Hi @mayuri18kadam@gmail.com ,
This could be a limitation from Spark submit job. Please check the docs from here https://docs.databricks.com/jobs.html#create-a-job please look for the following information:
Important
There are several limitations for spark-submit tasks....
