Best Practices for Optimizing Delta Lake Performance

[joshbuttler]

I’m currently working on a project using Delta Lake on Databricks and would like to optimize performance for both batch and streaming workloads.

[szymon_dybczak]

Hi @joshbuttler ,

It's a big topic and it really depends on the use case. You can follow some guides/recommendations from databricks documentation:

- Optimization recommendations on Databricks | Databricks on AWS

- Use liquid clustering for Delta tables | Databricks on AWS

- Comprehensive Guide to Optimize Data Workloads | Databricks

And if your company is databricks partner then you have access to a nice course on optimization in Databricks Academy for free.