Transition from partitioned table to Liquid clustered table
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
โ02-19-2025 11:07 PM
Hi all,
I have a table called classes, which is already partitioned on three different columns. I want to create a Liquid Clustered Table, but as far as I understand from the documentationโand from Dany Lee and his teamโit was not possible as of 2024 YouTube Video.
I believe there is still no smooth transition. So, I want to recreate the table, but is the only way to do this by creating a new table, deleting the old one, and moving everything from new_table to the old tableโs location?
%sql
-- Step1
CREATE TABLE IF NOT EXISTS class_clustered
CLUSTER BY (id, family, result_date)
LOCATION 'my_new_table_location'
AS
SELECT *
FROM class(the main partitioned table);
-- Step2
DROP class
-- Step3
Remove location in dbfs
-- Step4
Move everything from location my_new_table_location --> old location(for class partitioned)
Can I overwrite it or do something similar without too much effort? ๐ There are more than 20 tables that need to be converted to Liquid Clustered Tables, so Iโm looking for the best approach.
- Labels:
-
Delta Lake
-
Spark
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
โ02-20-2025 12:04 AM
You can enable liquid clustering on an existing table:
ALTER TABLE <table_name>
CLUSTER BY (<clustering_columns>)
You need to run OPTIMIZE FULL to force reclustering.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
โ02-20-2025 12:08 AM - edited โ02-20-2025 12:10 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
โ02-20-2025 12:13 AM
I'm sorry, I missed that your tables are partitioned, in this case easiest way will be to recreate it.

