You can find some tips here: https://community.databricks.com/t5/technical-blog/top-10-query-performance-tuning-tips-for-databricks-serverless/ba-p/43218 And here: https://www.databricks.com/discover/pages/optimize-data-workloads-guide
My overall re...
An idea, not sure if it helps you... You could set up a new task dependent on every other task, but with the run option set to "Run If at least one dependency failed". This new task would be a simple API call to the CANCEL RUN or CANCEL ALL RUNS OF A...
Notebooks are stored in the client's cloud storage. The diagram might have confused you, but it just goes to show that you access the notebooks via control plane, not that they're stored there. Azure Databricks automatically creates the workspace sto...
Hey @Phani1 !
I have had a few cases where we have seen huge improvement when moving to SQL Serverless. Processes that used to take hours in Synapse went to minutes with Serverless. Unfortunately i don't have cost benchmarks.
About use cases, Datab...
