topic Re: Liquid clustering vs partitioning in Data Engineering

Liquid clustering vs partitioning

Avinash_Narala — Wed, 17 Jul 2024 10:20:34 GMT

Hi,

Is liquid clustering a replacement to partitioning?

should we use still partitioning when we use liquid clustering?

Can we use liquid clustering for all cases and ignore partitioning?

Re: Liquid clustering vs partitioning

szymon_dybczak — Wed, 17 Jul 2024 10:38:46 GMT

Hi @Avinash_Narala

Yeah, you can think of it as a partitioning replacement.

According with documentation:

https://learn.microsoft.com/en-us/azure/databricks/delta/clustering

Delta Lake liquid clustering replaces table partitioning and ZORDER to simplify data layout decisions and optimize query performance. Liquid clustering provides flexibility to redefine clustering keys without rewriting existing data, allowing data layout to evolve alongside analytic needs over time.

Re: Liquid clustering vs partitioning

canadiandataguy — Thu, 13 Mar 2025 15:35:30 GMT

I have built a decision tree on how to think about it https://www.canadiandataguy.com/p/optimizing-delta-lake-tables-liquid?triedRedirect=true