difference between optimize and auto optimize and Optimize in delta
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-17-2021 01:27 AM
What would be good for me , if I should use Optimize every time or should I be using auto-optimize?
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-22-2021 04:07 PM
- Optimize: Bin-packing/Compaction. Idempotent and Incremental
- Optimize + Z-Order: Helps in Data Skipping; Use Range Partitioning
- Optimize write: Improve the write operation to the Delta table. optimization is performed before the write/during the write.
- Auto-compaction: Post-commit action Triggered when more than 50 files/directory
Auto-optimize = Optimize Write + Auto-compaction
What is suitable is determined by the use cases.
Reference: