โ03-14-2022 03:55 AM
getting error as below while creating buckets on delta table.
Error in SQL statement: AnalysisException: Delta bucketed tables are not supported.
have fall back to parquet table due to this for some use cases. is their any alternative for this. i have to join over 30 tables and biggest table have over 300M Records . joining fields have high cardinality and can't be used for partition so need to do bucketing.
โ05-10-2022 05:57 AM
Hi @Rahul Samantโ , we checked internally on this due to certain limitations bucketing is not supported on delta tables, the only alternative for bucketing is to leverage the z ordering, below is the link for reference
โ03-15-2022 12:36 AM
Delta has Z-Ordering and Bloom filters, check it out, lots to read:
โ03-15-2022 12:46 AM
@Werner Stinckensโ , Yeah checked it but Z-Ordering has to be triggered separately and it doesn't work well with auto compaction enabled with delta table.
โ04-27-2022 12:57 AM
Hi @Rahul Samantโ , Just a friendly follow-up. Do you still need help, or @Werner Stinckensโ 's response help you to find the solution? Please let us know.
โ04-27-2022 01:05 AM
Thanks @Kaniz Fatmaโ for the follow-up , we are using mix of both types. depending on the use-case.
didn't find the alternative for bucketing
โ05-10-2022 05:57 AM
Hi @Rahul Samantโ , we checked internally on this due to certain limitations bucketing is not supported on delta tables, the only alternative for bucketing is to leverage the z ordering, below is the link for reference
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโt want to miss the chance to attend and share knowledge.
If there isnโt a group near you, start one and help create a community that brings people together.
Request a New Group