Hi @vijaypodili 

ideally, given you have 8 cores (for 2 workers), repartition/numPartition should be a multiple of 8 (no. of cores).

Concern here is, I don't see any transformation in the code snippet shared that could potentially trigger a long running job. I strongly believe, writing it to the table in SSMS is taking longer in this case. 

For the job that's taking time to execute, would you be able to share the DAG from the spark UI?

Also, check the below StackOverFlow, where folks suggesting to use various connectors to improve write performance.