- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-05-2025 12:42 AM - edited 12-05-2025 12:44 AM
Hi @nanditakrishnan ,
There's already something like that in databricks dashboards, but some conditions need to be fulfilled (i.e queries need to share same group by). One of dataset optimization techniques that databricks team implemented is doing following:
"For visualization queries sent to the backend, separate visualization queries against the same dataset that share the same GROUP BY clauses and filter predicates are combined into a single query for processing. In this case, users may see a combined query in the query history that fetches results for multiple visualizations or filters."
There are also other optimization tricks that they're doing - you can read about them below:
Dataset optimization and caching - Azure Databricks | Microsoft Learn