Hubert-Dudek
Databricks MVP

Usually yes. It depends on partitioning. If you have 2 executors (writers) and every of them hold some partition which have to be append to delta, write process will be per partition simultaneously. You can also analyze you exact use case looking to jobs (and other tabs) in Spark UI.


My blog: https://databrickster.medium.com/