Issue with Auto Liquid clustering

RevanthV
Contributor

‎09-13-2025 12:58 AM

 

  • I have written data to a table using clusterByAuto set to true
  • But the clustering keys are not selected automatically when i do a desc detail on the table.Screenshot below

    Why are clustering columns not being selected automatically?

Repro steps:

  1. Create a dataframe
  2. Write to a delta table using clusterByAuto set to true
  3. Then run desc detail on teh table .You will not see any clustering columns selected

K_Anudeep
Databricks Employee
Databricks Employee

‎09-13-2025 01:42 AM

Hello Revanth,

It could be that the 

  • The table is too small to benefit from liquid clustering, or it has a good clustering scheme. Could you tell me the size of the table?
  • Also, if the table is not being frequently queried on a column or set of columns,ie.., there are not enough scans on the table, so it wouldn't benefit from clustering
  • Also, Auto Liquid works with PO in the background, so based on some internal thresholds, the clustering columns are selected, and we as Cx need not worry about that 

So once you have frequent scans on the table columns, you will see clustering columns being selected automatically.

You can go through the doc for better understanding: https://docs.databricks.com/aws/en/delta/clustering#:~:text=If%20a%20key%20was%20not%20selected%20by...

Let me know if you have any further questions.

Anudeep

View solution in original post

RevanthV
Contributor
In response to K_Anudeep

‎09-13-2025 04:51 AM

Thanks a lot @K_Anudeep , My table is still small and i guess that was the reason ,  have now written around 1million records and have been just running frequent scans on a column since last one hour and now i can see the same column selected as a clustering column.

Thanks a lot for your help

szymon_dybczak
Esteemed Contributor III

‎09-13-2025 02:04 AM

Hi @RevanthV ,

As @K_Anudeep  correctly suggested it could be the case that your table is to small to benefit from liquid clustering.
Another possibility it that you're using runtime lower than 15.4 LTS.