What is the difference between the Auto Optimize set on Spark Session to the one set on Delta Table.

Madhur
Databricks Partner
 

jose_gonzalez
Databricks Employee
Databricks Employee

Hi @Madhur,

The difference between Auto Optimize set on Spark Session and the one set on Delta Table lies in their scope and precedence.

  • Auto Optimize on Spark Session will apply to all Delta tables in the current session. It is a global configuration which takes precedence over the related table property during the current session.
  • Auto Optimize on Delta Table, applies only to the specific Delta table on which it is set. If a property is set on a table, then this is the setting that is followed by default unless overridden by a SparkSession configuration.

View solution in original post