What file statistics does Optimize return & How can I use them to my advantage?
Anonymous
Not applicable
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-22-2021 07:07 PM
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-22-2021 08:24 PM
Optimize is largely designed as a data organization strategy for Delta Tables. Its help by compacting small files, collecting columns stats to help with data skipping and also Z-ordering of data if that's called explicitly can help with both read/write performance on Delta Tables. See this for additional details - https://docs.databricks.com/delta/optimizations/file-mgmt.html