Spark read not working in parallel

ssaloni_bhatia
New Contributor

Hi All,

Am trying to run a spark jdbc query and the read is running on only 1 worker and is taking a lot of time. Usually, it takes 4 minutes in aws lambda but now it is taking approx 10 minutes while using databricks. There are only 4m records (write is not taking long but read is consuming the max time)