Hi @ShaliniC ,

The possible causes are:
1. The Databricks Runtime version of the job cluster is different than the one of the the shared cluster.

2. The libraries installed on the shared cluster are missing in the job cluster.

Could you check the what are the above configuration changes and adjust the job cluster to the configuration of the shared cluster?

If it does not help, could you please share the details of the error message?