Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-23-2022 01:56 AM
Please increase parallelism by adjusting jdbc settings:
columnName="key",
lowerBound=1L,
upperBound=100000L,
numPartitions=100,
It is example values. The best that key column would be unique and continuous so it will be divided equally without data skews.
Please analyze also Spark UI - look what takes the biggest time (reading or writing?)
My blog: https://databrickster.medium.com/