Spark read not working in parallel

[ssaloni_bhatia]

Hi All,

Am trying to run a spark jdbc query and the read is running on only 1 worker and is taking a lot of time. Usually, it takes 4 minutes in aws lambda but now it is taking approx 10 minutes while using databricks. There are only 4m records (write is not taking long but read is consuming the max time)

[Anonymous]

Hi @Saloni Bhatia​ 

Great to meet you, and thanks for your question!

Let's see if your peers in the community have an answer to your question. Thanks.