โ03-14-2022 03:55 AM
getting error as below while creating buckets on delta table.
Error in SQL statement: AnalysisException: Delta bucketed tables are not supported.
have fall back to parquet table due to this for some use cases. is their any alternative for this. i have to join over 30 tables and biggest table have over 300M Records . joining fields have high cardinality and can't be used for partition so need to do bucketing.
โ05-10-2022 05:57 AM
Hi @Rahul Samantโ , we checked internally on this due to certain limitations bucketing is not supported on delta tables, the only alternative for bucketing is to leverage the z ordering, below is the link for reference
โ03-15-2022 12:36 AM
Delta has Z-Ordering and Bloom filters, check it out, lots to read:
โ03-15-2022 12:46 AM
@Werner Stinckensโ , Yeah checked it but Z-Ordering has to be triggered separately and it doesn't work well with auto compaction enabled with delta table.
โ04-27-2022 12:57 AM
Hi @Rahul Samantโ , Just a friendly follow-up. Do you still need help, or @Werner Stinckensโ 's response help you to find the solution? Please let us know.
โ04-27-2022 01:05 AM
Thanks @Kaniz Fatmaโ for the follow-up , we are using mix of both types. depending on the use-case.
didn't find the alternative for bucketing
โ05-10-2022 05:57 AM
Hi @Rahul Samantโ , we checked internally on this due to certain limitations bucketing is not supported on delta tables, the only alternative for bucketing is to leverage the z ordering, below is the link for reference