Liquid clustering on and dynamic overwrites
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-11-2024 03:37 AM
I use the following option to write from multiple tasks to the same table with overwrite (in Pyspark)
.option("partitionOverwriteMode", "dynamic")The table was created with partition by so it works as expected.
I read about liquid clustering and it's benefits over table partitioning
Is it possible to perform the same use case of writing from multiple tasks (simultaneously) to the same table with overwrite on a liquid clustered table?
Thanks
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-03-2024 04:42 AM
No it doesn't support partition overwrite.