topic Re: Transition from partitioned table to Liquid clustered table in Data Engineering

Transition from partitioned table to Liquid clustered table

mjedy78 — Thu, 20 Feb 2025 07:07:58 GMT

Hi all,
I have a table called classes, which is already partitioned on three different columns. I want to create a Liquid Clustered Table, but as far as I understand from the documentation—and from Dany Lee and his team—it was not possible as of 2024 YouTube Video.

I believe there is still no smooth transition. So, I want to recreate the table, but is the only way to do this by creating a new table, deleting the old one, and moving everything from new_table to the old table’s location?

%sql -- Step1 CREATE TABLE IF NOT EXISTS class_clustered CLUSTER BY (id, family, result_date) LOCATION 'my_new_table_location' AS SELECT * FROM class(the main partitioned table); -- Step2 DROP class -- Step3 Remove location in dbfs -- Step4 Move everything from location my_new_table_location --> old location(for class partitioned)

Can I overwrite it or do something similar without too much effort? 😃 There are more than 20 tables that need to be converted to Liquid Clustered Tables, so I’m looking for the best approach.

Re: Transition from partitioned table to Liquid clustered table

MariuszK — Thu, 20 Feb 2025 08:04:59 GMT

You can enable liquid clustering on an existing table:

ALTER TABLE <table_name>
CLUSTER BY (<clustering_columns>)

You need to run OPTIMIZE FULL to force reclustering.

Re: Transition from partitioned table to Liquid clustered table

mjedy78 — Thu, 20 Feb 2025 08:10:04 GMT

@MariuszK It is for normal tables and will not work with the partitioned ones

Re: Transition from partitioned table to Liquid clustered table

MariuszK — Thu, 20 Feb 2025 08:13:37 GMT

I'm sorry, I missed that your tables are partitioned, in this case easiest way will be to recreate it.

Re: Transition from partitioned table to Liquid clustered table

biancaorita — Thu, 09 Apr 2026 16:23:36 GMT

Is there a plan to implement a way to migrate to liquid clustering for an existing table that has traditional partitioning and that is quite large (over 4 TB)? Re-creating such tables from scratch is not always ideal.