Databricks Predictive optimization
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-11-2024 10:54 PM
If we want to enable Databricks Predictive Optimization, then is it also mandatory to enable serverless Job/Notebook Compute in our account. We already have Serverless SQL warehouse available in our workspaces.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-12-2024 12:04 AM
There is no specific requirement to have Serverless to enable Predictive Optimization.
You must fulfill the following requirements to enable predictive optimization:
-
Your Databricks workspace must be on the Premium plan or above in a region that supports predictive optimization. See Databricks clouds and regions.
-
You must use SQL warehouses or Databricks Runtime 12.2 LTS or above when you enable predictive optimization.
-
Only Unity Catalog managed tables are supported.
Please refer to this doc: https://docs.databricks.com/en/optimizations/predictive-optimization.html
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-13-2024 02:38 AM
Thanks for your answer, One more question on this, I think in case to use predictive optimization we should first enable this at account level ? If this is the case then by doing this each of the catalogue/schema/table in Account will start using predictive optimization? should we first disable this for list of catalogues where we do not want this to be used using "ALTER CATALOG [catalog_name] DISABLE PREDICTIVE OPTIMIZATION;"" before enabling at Account level ?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-13-2024 10:14 PM
Thanks. This was answered in https://community.databricks.com/t5/administration-architecture/enable-predictive-optimization/td-p/...
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
a week ago
The documentation states this:
Predictive optimization identifies tables that would benefit from ANALYZE, OPTIMIZE, and VACUUM operations and queues them to run using serverless compute for jobs.
If I don't have serverless workloads enabled how does predictive optimization optimizes the tables?

