Best Practices for Optimizing Delta Lake Performance

joshbuttler — Wed, 18 Sep 2024 08:18:03 GMT

I’m currently working on a project using Delta Lake on Databricks and would like to optimize performance for both batch and streaming workloads.

Re: Best Practices for Optimizing Delta Lake Performance

szymon_dybczak — Wed, 18 Sep 2024 11:25:59 GMT

It's a big topic and it really depends on the use case. You can follow some guides/recommendations from databricks documentation:

- Optimization recommendations on Databricks | Databricks on AWS

And if your company is databricks partner then you have access to a nice course on optimization in Databricks Academy for free.