- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-04-2024 12:51 AM
@DatBoi
Once you watch this video you'll understand more about Liquid Clustering 🙂
https://www.youtube.com/watch?v=5t6wX28JC_M&ab_channel=DeltaLake
Long story short:
I know Databricks recommends not partitioning on tables less than 1 TB and aim for 1 GB partitions. Does this hold true for liquid clustering?
Clustering is a little bit different from partitioning. The main issue with partitioning tables less than 1TB is that it could create a lot of small files, that could negatively impact performance. With liquid clustering there's no such issue.
And vice versa - will clustering on a small table < 1TB or even < 1GB hinder the performance of queries?
From my experience - not really.