when automatic liquid clustering is enabled, how to know which columns are used for clustering?

[smoortema]

Let's say a table is configured to have automatic liquid clustering:

ALTER TABLE table1 CLUSTER BY AUTO;

 How to know which columns were chosen by Databricks?

[szymon_dybczak · Accepted solution]

Hi @smoortema ,

You can use DESCRIBE TABLE or SHOW TBLPROPERTIES. If automatic liquid clustering is enabled, the clusterByAuto property is set to true. The clusteringColumns property shows the current clustering columns that were automatically or manually selected.

Use liquid clustering for tables | Databricks on AWS

[smoortema]

Is there a way to set up automatic clustering by giving it a hint to use certain columns until Databricks has enough history to choose one?

[szymon_dybczak]

Unfortunately there is no option for doing that currently. But it seems like a really good idea for a product team to consider 🙂

[smoortema]

From the documentation, it seems that in Python, there is such an option, only when creating or replacing a table.

# To set clustering columns and auto, which serves as a way to give a hint
# for the initial selection.
df.writeTo(...).using("delta")
  .clusterBy("clusteringColumn1", "clusteringColumn2")
  .option(“clusterByAuto”, “true”)
  .create()

[szymon_dybczak]

Thanks for letting me know. I wasn't aware of that 🙂