topic Re: Enable Predictive optimization in Administration & Architecture

Enable Predictive optimization

AnkurMittal008 — Thu, 14 Nov 2024 05:24:16 GMT

In case to use predictive optimization we should first enable this at account level ? If this is the case then by doing this each of the catalogue/schema/table in Account will start using predictive optimization by default? should we first disable this for list of catalogues where we do not want this to be used using "ALTER CATALOG [catalog_name] DISABLE PREDICTIVE OPTIMIZATION;"" before enabling at Account level ?

Re: Enable Predictive optimization

SparkJun — Thu, 14 Nov 2024 06:13:15 GMT

Yes, you must enable predictive optimization at the account level. Once enabled, predictive optimization will be applied by default to all catalogs, schemas, and tables within the account. To prevent predictive optimization from being applied to specific catalogs, you should disable it for those catalogs before enabling it at the account level. You can do this using the following command:

ALTER CATALOG [catalog_name] DISABLE PREDICTIVE OPTIMIZATION;

This way, when you enable predictive optimization at the account level, it will not be applied to the catalogs where you have explicitly disabled it.

Ref. https://docs.databricks.com/en/optimizations/predictive-optimization.html

Re: Enable Predictive optimization

AnkurMittal008 — Mon, 18 Nov 2024 08:18:14 GMT

Thanks a lot @SparkJun , In documentation I am not able to find answer to one scenario

Let's say we have explicitly disable predictive optimization for a Catalog named "CatalogXYZ" and then after that we have enabled this at Account level. Later a user who is the owner of Schema named "CatalogXYZ.TestSchema" can enable the predictive optimization for this schema or not because it was disabled explicitly for parent Catalog?