- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-15-2023 11:07 PM
Hi,
Do you know any good resources about Databricks performance improvements(like improving query performances, monitoring/resolving performance bottlenecks etc)?
Thanks
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-16-2023 12:48 AM
Hi @Ömer Özsakarya I am sharing few official documentation and Kb's for query performance improvements and few best practices to avoid bottlenecks.
https://docs.databricks.com/optimizations/index.html
https://docs.databricks.com/sql/admin/query-caching.html
https://docs.databricks.com/optimizations/disk-cache.html
https://docs.databricks.com/lakehouse-architecture/performance-efficiency/best-practices.html
https://kb.databricks.com/en_US/scala/jobs-fails-with-a-timeoutexception-error
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-16-2023 04:20 AM
In addition to Vinay's answer, you should also check for the Best practices for cluster configurations as clusters play an important role in improving job performance.
https://docs.databricks.com/clusters/cluster-config-best-practices.html
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-17-2023 12:18 AM
Hi @Ömer Özsakarya
We haven't heard from you since the last response from @Lakshay Goel , and I was checking back to see if her suggestions helped you.
Or else, If you have any solution, please share it with the community, as it can be helpful to others.
Also, Please don't forget to click on the "Select As Best" button whenever the information provided helps resolve your question.
