difference between optimize and auto optimize and Optimize in delta

[User16826994223]

What would be good for me , if I should use Optimize every time or should I be using auto-optimize?

[User16869510359]

• Optimize:  Bin-packing/Compaction. Idempotent and Incremental
• Optimize + Z-Order: Helps in Data Skipping; Use Range Partitioning
• Optimize write: Improve the write operation to the Delta table. optimization is performed before the write/during the write.
• Auto-compaction: Post-commit action Triggered when more than 50 files/directory

Auto-optimize = Optimize Write + Auto-compaction

What is suitable is determined by the use cases.

Reference:

• https://docs.databricks.com/delta/optimizations/auto-optimize.html
• https://docs.databricks.com/delta/optimizations/file-mgmt.html#data-skipping