Dicer
Valued Contributor

I tried the following one and it still took more than 10 hours until Fatal error: The Python kernel is unresponsive.

%sql
 
--Enable Auto Optimization
set spark.databricks.delta.properties.defaults.autoOptimize.optimizeWrite = true;
set spark.databricks.delta.properties.defaults.autoOptimize.autoCompact = true;