03-14-2022 03:55 AM
getting error as below while creating buckets on delta table.
Error in SQL statement: AnalysisException: Delta bucketed tables are not supported.
have fall back to parquet table due to this for some use cases. is their any alternative for this. i have to join over 30 tables and biggest table have over 300M Records . joining fields have high cardinality and can't be used for partition so need to do bucketing.
05-10-2022 05:57 AM
Hi @Rahul Samant , we checked internally on this due to certain limitations bucketing is not supported on delta tables, the only alternative for bucketing is to leverage the z ordering, below is the link for reference
03-15-2022 12:36 AM
Delta has Z-Ordering and Bloom filters, check it out, lots to read:
03-15-2022 12:46 AM
@Werner Stinckens , Yeah checked it but Z-Ordering has to be triggered separately and it doesn't work well with auto compaction enabled with delta table.
04-27-2022 12:57 AM
Hi @Rahul Samant , Just a friendly follow-up. Do you still need help, or @Werner Stinckens 's response help you to find the solution? Please let us know.
04-27-2022 01:05 AM
Thanks @Kaniz Fatma for the follow-up , we are using mix of both types. depending on the use-case.
didn't find the alternative for bucketing
05-10-2022 05:57 AM
Hi @Rahul Samant , we checked internally on this due to certain limitations bucketing is not supported on delta tables, the only alternative for bucketing is to leverage the z ordering, below is the link for reference
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
Click here to register and join today!
Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.