https://community.databricks.com/t5/data-engineering/is-there-any-way-to-control-the-autooptimize-interval/m-p/19863#M13390 <P>I can see my streaming jobs running optimize jobs more frequently, Is there any property so I can control autoOptimize duration</P> Fri, 25 Jun 2021 13:19:20 GMT brickster_2018 2021-06-25T13:19:20Z https://community.databricks.com/t5/data-engineering/is-there-any-way-to-control-the-autooptimize-interval/m-p/19863#M13390 <P>I can see my streaming jobs running optimize jobs more frequently, Is there any property so I can control autoOptimize duration</P> Fri, 25 Jun 2021 13:19:20 GMT https://community.databricks.com/t5/data-engineering/is-there-any-way-to-control-the-autooptimize-interval/m-p/19863#M13390 brickster_2018 2021-06-25T13:19:20Z https://community.databricks.com/t5/data-engineering/is-there-any-way-to-control-the-autooptimize-interval/m-p/19864#M13391 <P>The autoOptimize is not performed on a time basis. It's an event-based trigger. Once the delta table/partition has 50 (default value of spark.databricks.delta.autoCompact.minNumFiles) files, auto-compaction is triggered. </P><P>To reduce the frequency, increase the value of the aforementioned configuration. optionally you can completely turn off the auto-compaction and run a regular optimize command on a daily basis. </P> Fri, 25 Jun 2021 13:23:21 GMT https://community.databricks.com/t5/data-engineering/is-there-any-way-to-control-the-autooptimize-interval/m-p/19864#M13391 brickster_2018 2021-06-25T13:23:21Z