cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

What is the best practice while loading delta table , do I infer the schema or provide the schema?

palzor
New Contributor III

I am loading avro files into the detla tables. I am doing this for multiple tables and some files are big like (2-3GB) and most of them are small like in few MBs.

I am using autoloader to load the data into the delta tables.

My question is:

What is the best practice , do I infer the schema or provide the schema while using autoloader.

I understand that if I provide the schema , the processing would be faster however it would be time consuming to provide the schema for tables that has many columns.

Any suggestions?

0 REPLIES 0
Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.