So firstly, each task has a separate cluster when it comes to spark-submit jobs, so for those jobs, it is the same behavior across all jobs in the dev workspace that we have.

in the prod workspace, this issue is there for some tasks and not there for other tasks