Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-26-2024 01:02 PM
Yes,
First of all, open source spark already has a set of auto-tuning features denominated Adaptive Query Execution (AQE). Here are more details: https://spark.apache.org/docs/latest/sql-performance-tuning.html#adaptive-query-execution.
For even better auto-tuning capabilities, Databricks offer it's Photon Engine that is a highly optimized engine for spark that explores spark's AQE features much further than the OSS spark engine.
Databricks SQL is also a highly optimized product that doesn't need manual tuning of your workloads. DBSQL relies on Photon.
Best regards,
Raphael Balogo
Sr. Technical Solutions Engineer
Databricks
Raphael Balogo
Sr. Technical Solutions Engineer
Databricks