topic Unable to configure clustering on DLT tables in Data Engineering

Unable to configure clustering on DLT tables

Chris_N — Fri, 10 Oct 2025 19:04:57 GMT

Hi Team

I have a DLT pipeline with `cluster_by` property configured for all my tables. The code looks something like below:

@Dlt.table( name="flows", cluster_by=["from"] ) def flows(): <LOGIC>

It was all working fine and in couple of days, the queries were taking forever and when I checked my dlt tables. I couldn't find any cluster properties configured. I tried setting 'cluster_by_auto=True' and it was properly configured but the cluster columns are not taken into consideration.

Is this some bug with the latest release or is there a way to solve this?

Thanks in advance

Re: Unable to configure clustering on DLT tables

NandiniN — Fri, 10 Oct 2025 21:40:45 GMT

Hi @Chris_N ,

You have mentioned - "I couldn't find any cluster properties configured."

If they existed and were changed, you can use the delta history command to check if someone changed on the clustering information.

It is possible there were changes in the data volume/configs that could have led to the change in performance.

On the next statement of yours - " I tried setting 'cluster_by_auto=True' and it was properly configured but the cluster columns are not taken into consideration." - How do you determine this that, the cluster columns are not considered?

After liquid clustering is enabled, run OPTIMIZE jobs as usual to incrementally cluster data. See How to trigger clustering.

Liquid clustering is incremental, meaning that data is only rewritten as necessary to accommodate data that needs to be clustered. Data files with clustering keys that do not match the data to be clustered are not rewritten.

Please let me know of additional informations, so that I can suggest further. Additionally, if you have a support subscription, a support ticket can also be raised to investigate this.

Thanks & Regards,

Nandini

Re: Unable to configure clustering on DLT tables

Chris_N — Sat, 11 Oct 2025 10:20:42 GMT

There was no changes to the clustering configuration of the DLT table (Materialized view).

I am checking if the clustering fields are properly configured using the SQL command `DESCRIBE EXTENDED <table>`. While running this on a normal table with clustering configured, I can see `clusteringColumns` property under the TableProperties section.

The same does not exist for the DLT tables created. This issue exists for all my DLT tables across different pipelines.

For recreating the issue, please create a DLT table using the below syntax and check if the clusteringColumns property is properly set on the table

@Dlt.table( name="flows", cluster_by=["from"] ) def flows(): <LOGIC>

Re: Unable to configure clustering on DLT tables

saurabh18cs — Mon, 13 Oct 2025 11:36:37 GMT

Hi @Chris_N maybe cluster_by for dlt's does not persist in delta table metadata properties as this is a hint for dlt pipelines and thus you cannot see it.

As other have confirmed if your tables are not managed tables and you are not using predective optimization then you have to make sure to optimize/z-order tables regularly as a separate workflow/job to make your querties run faster.

Note: Over time, as new data is appended and files are rewritten, the clustering effect can diminish unless you regularly OPTIMIZE/Z-ORDER the table.