Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-23-2021 06:28 PM
A better way I can think of is-
Enable auto optimize (it will automatically create a file of 128 mb)
Enable Auto compact.
delta.autoOptimize.optimizeWrite = true
delta.autoOptimize.autoCompact = true complete guide-
https://docs.microsoft.com/en-us/azure/databricks/delta/optimizations/auto-optimize