Hello @felix_counter ,

Thank you for providing the details.

The "auto OPTIMIZE" you're referring to is not automatically triggered by Databricks. Instead, it seems to be initiated by an external service. The only features inside Databricks that trigger OPTIMIZE operations at the current moment are Predictive optimization for Delta Lake , Maintenance tasks performed by Delta Live Tables and Auto compaction for Delta Lake on Databricks . 

You are correct in your understanding that the 'spark.databricks.delta.optimizeWrite.enabled' setting performs an on-write operation, which does not register as an OPTIMIZE operation in the history.

 

 

Best regards,

Raphael Balogo
Sr. Technical Solutions Engineer
Databricks