Hi @borori,

How are you doing today?

As per my understanding, Consider checking the cluster's resource limits in serverless mode to ensure it's not hitting any memory or I/O constraints. You might also want to repartition the DataFrame based on the date column before writing to balance the load across partitions. It could be helpful to examine Delta logs to see if they provide any insights on the issue during the write process. Also, review your partitioning strategy—too many or too few partitions can affect performance. Lastly, try adjusting the job's parallelism settings by tuning parameters like spark.sql.shuffle.partitions to improve the write performance.

Give a try and let me know if it works.

Regards,

Brahma