cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Transition from partitioned table to Liquid clustered table

mjedy78
New Contributor II

Hi all,
I have a table called classes, which is already partitioned on three different columns. I want to create a Liquid Clustered Table, but as far as I understand from the documentationโ€”and from Dany Lee and his teamโ€”it was not possible as of 2024 YouTube Video.

I believe there is still no smooth transition. So, I want to recreate the table, but is the only way to do this by creating a new table, deleting the old one, and moving everything from new_table to the old tableโ€™s location?

 

%sql
-- Step1
CREATE TABLE IF NOT EXISTS class_clustered
CLUSTER BY (id, family, result_date)
LOCATION 'my_new_table_location'
AS
SELECT *
FROM class(the main partitioned table);

-- Step2
DROP class

-- Step3
Remove location in dbfs

-- Step4
Move everything from location my_new_table_location --> old location(for class partitioned)

 

Can I overwrite it or do something similar without too much effort? ๐Ÿ˜ƒ There are more than 20 tables that need to be converted to Liquid Clustered Tables, so Iโ€™m looking for the best approach.

 

3 REPLIES 3

MariuszK
Valued Contributor III

You can enable liquid clustering on an existing table:

ALTER TABLE <table_name>
CLUSTER BY (<clustering_columns>)

You need to run OPTIMIZE FULL to force reclustering.

mjedy78
New Contributor II

@MariuszK It is for normal tables and will not work with the partitioned ones 

mjedy78_0-1740038996897.png

 

 

MariuszK
Valued Contributor III

I'm sorry, I missed that your tables are partitioned, in this case easiest  way will be to recreate it.