Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-22-2022 09:37 AM
Please extend the number of dataframe partitions using
coalesce(<N>) or repartition(<N>). In most cases, it should save the issue automatically as it will write in chunks per partition.
In addition these jdbc connection properties can help (as on JDBC To Other Databases - Spark 3.3.0 Documentation (apache.org)😞
numPartitions
batchsize
isolationLevel
My blog: https://databrickster.medium.com/