topic Re: DLT maintainace clusters in Data Engineering

DLT maintainace clusters

Faisal — Wed, 11 Oct 2023 11:07:52 GMT

How does maintenance clusters do the cleanup using optimize, zorder and vacuum. I read that it is handled automatically but how does maintenance cluster know which column to optimize, where do we need to specify that info ?

Re: DLT maintainace clusters

Faisal — Mon, 16 Oct 2023 10:46:07 GMT

Thanks @Retired_mod for the response,can you please give me complete syntax of "use zorder". Also do I need to create maintenance cluster or is it automatically done when specifying dlt policy?

Re: DLT maintainace clusters

cgrant — Wed, 18 Dec 2024 23:46:52 GMT

At this time, Z-order columns must be specified in the asset definition, the property is pipelines.autoOptimize.zOrderCols. This may change in the future with Predictive Optimization.