jose_gonzalez
Databricks Employee
Databricks Employee

Hi @Franco Sia​ ,

I will recommend to avoid to use the repartition(50), instead enable optimizes writes on your Delta table. You can find more details here

Enable optimized writes and auto compaction on your Delta table. Use AQE (docs here) to have enough shuffle partitions and also check your joins in case you have any data skew.