Bucketing on Delta Tables

[Rahul_Samant]

getting error as below while creating buckets on delta table.

Error in SQL statement: AnalysisException: Delta bucketed tables are not supported.

have fall back to parquet table due to this for some use cases. is their any alternative for this. i have to join over 30 tables and biggest table have over 300M Records . joining fields have high cardinality and can't be used for partition so need to do bucketing.

[-werners-]

Delta has Z-Ordering and Bloom filters, check it out, lots to read:

https://docs.databricks.com/delta/optimizations/index.html

[Rahul_Samant]

@Werner Stinckens​ , Yeah checked it but Z-Ordering has to be triggered separately and it doesn't work well with auto compaction enabled with delta table.

[Kaniz_Fatma]

Hi @Rahul Samant​ , Just a friendly follow-up. Do you still need help, or @Werner Stinckens​ 's response help you to find the solution? Please let us know.

[Rahul_Samant]

Thanks @Kaniz Fatma​  for the follow-up , we are using mix of both types. depending on the use-case.

didn't find the alternative for bucketing

[Anonymous · Accepted solution]

Hi @Rahul Samant​  , we checked internally on this due to certain limitations bucketing is not supported on delta tables, the only alternative for bucketing is to leverage the z ordering, below is the link for reference

https://docs.databricks.com/delta/optimizations/file-mgmt.html#z-ordering-multi-dimensional-clusteri...