DLT maintainace clusters

Faisal
Contributor

How does maintenance clusters do the cleanup using optimize, zorder and vacuum. I read that it is handled automatically but how does maintenance cluster know which column to optimize, where do we need to specify that info ?

Faisal
Contributor

Thanks @Retired_mod for the response,can you please give me complete syntax of "use zorder". Also do I need to create maintenance cluster or is it automatically done when specifying dlt policy?

cgrant
Databricks Employee
Databricks Employee

At this time, Z-order columns must be specified in the asset definition, the property is pipelines.autoOptimize.zOrderCols. This may change in the future with Predictive Optimization.