cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Delta vs. Parquet

User16826992185
Databricks Employee
Databricks Employee

I'm curious about the benefits of using the Delta file format vs. Parquet. Is there any downside to using Delta?

1 REPLY 1

sean_owen
Databricks Employee
Databricks Employee

Not really. You get upsides like transactions, time travel, upsert/merge/deletes. There is some cost to that, as Delta manages that by writing and managing many smaller Parquet files and has to re-read them to recreate the current or past state of the data. VACUUMing the data set periodically takes time too. So you may incur a little runtime overhead for these reasons; then again, Delta offers advanced features like z-order indexing and data skipping with Spark that also make it faster to read than Parquet.

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now