topic Re: Databricks Predictive optimization in Administration & Architecture

Databricks Predictive optimization

AnkurMittal008 — Tue, 12 Nov 2024 06:54:50 GMT

If we want to enable Databricks Predictive Optimization, then is it also mandatory to enable serverless Job/Notebook Compute in our account. We already have Serverless SQL warehouse available in our workspaces.

Re: Databricks Predictive optimization

MuthuLakshmi — Tue, 12 Nov 2024 08:04:22 GMT

@AnkurMittal008

There is no specific requirement to have Serverless to enable Predictive Optimization.
You must fulfill the following requirements to enable predictive optimization:

Your Databricks workspace must be on the Premium plan or above in a region that supports predictive optimization. See Databricks clouds and regions.
You must use SQL warehouses or Databricks Runtime 12.2 LTS or above when you enable predictive optimization.
Only Unity Catalog managed tables are supported.

Please refer to this doc: https://docs.databricks.com/en/optimizations/predictive-optimization.html

Re: Databricks Predictive optimization

AnkurMittal008 — Wed, 13 Nov 2024 10:38:01 GMT

Thanks for your answer, One more question on this, I think in case to use predictive optimization we should first enable this at account level ? If this is the case then by doing this each of the catalogue/schema/table in Account will start using predictive optimization? should we first disable this for list of catalogues where we do not want this to be used using "ALTER CATALOG [catalog_name] DISABLE PREDICTIVE OPTIMIZATION;"" before enabling at Account level ?

Re: Databricks Predictive optimization

SparkJun — Thu, 14 Nov 2024 06:14:53 GMT

Thanks. This was answered in https://community.databricks.com/t5/administration-architecture/enable-predictive-optimization/td-p/98731

Re: Databricks Predictive optimization

htd350 — Wed, 09 Apr 2025 13:32:10 GMT

The documentation states this:

Predictive optimization identifies tables that would benefit from ANALYZE, OPTIMIZE, and VACUUM operations and queues them to run using serverless compute for jobs.

If I don't have serverless workloads enabled how does predictive optimization optimizes the tables?