cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Predictive Optimization keeps running on system.lakebase after disabling it at account level

trantu
Visitor

Hi all,

On an Azure Databricks workspace (Premium, Southeast Asia) that uses Lakebase, Predictive Optimization started running on 16 Nov 2026 on the Databricks-managed telemetry tables in system.lakebase (postgres_logs, compute_counters, plan_history and 7 others). It runs VACUUM, DELETE and PURGE every day and is billed as billing_origin_product = PREDICTIVE_OPTIMIZATION (SKU PREMIUM_JOBS_SERVERLESS_COMPUTE). We have no tables of our own that use it.

What we tried:
1. Disabled Predictive Optimization at account level (Account Console > Settings > Feature enablement). The metastore and the system catalog now show DISABLE.
2. But the schema system.lakebase still has enable_predictive_optimization = ENABLE set explicitly, and its tables inherit it. Operations kept running after the change (per system.storage.predictive_optimization_operations_history).
3. ALTER SCHEMA system.lakebase DISABLE PREDICTIVE OPTIMIZATION and the Unity Catalog API (schemas update) both fail with: PERMISSION_DENIED: Predictive optimization can't be modified on a system schema.

Usage so far: 2.58, 4.08 and 1.72 DBU per day. Small, but for a small R&D account it is the largest item on the bill.

Questions:
- Is there a supported way to turn Predictive Optimization off for system.lakebase, or to make it honour the account-level setting?
- Is this maintenance of system telemetry tables expected to be billed to the customer?

Thanks!

0 REPLIES 0