Hubert-Dudek
Databricks MVP

Please extend the number of dataframe partitions using  

coalesce(<N>) or repartition(<N>). In most cases, it should save the issue automatically as it will write in chunks per partition.

In addition these jdbc connection properties can help (as on JDBC To Other Databases - Spark 3.3.0 Documentation (apache.org)😞

numPartitions

batchsize

isolationLevel


My blog: https://databrickster.medium.com/

View solution in original post