ZCube Tags not present in Databricks Delta Tables

Mithos — Sat, 07 Dec 2024 19:21:01 GMT

The design doc for Liquid Clustering for Delta refer to Z-Cube to enable incremental clustering in batches. This is the link - https://docs.google.com/document/d/1FWR3odjOw4v4-hjFy_hVaNdxHVs4WuK1asfB6M6XEMw/edit?pli=1&tab=t.0.

It is also mentioned that the Z-Cube ID will be added as a tag to the transaction log. I could not locate a Z-Cube ID in the transaction logs for a Liquid Clustered table produced in Databricks. Is the Databricks Delta Liquid Clustering approach to incremental clustering different.

#Liquid Clustering #Databricks

Re: ZCube Tags not present in Databricks Delta Tables

VZLA — Wed, 11 Dec 2024 18:11:45 GMT

Hi @Mithos thanks for the question!

This is the OSS version of LC applicable to OSS Delta. Databricks has a different implementation, so you won't be able to find it in a liquid table written by DBR.

topic Re: ZCube Tags not present in Databricks Delta Tables in Data Engineering

ZCube Tags not present in Databricks Delta Tables

Re: ZCube Tags not present in Databricks Delta Tables