stbjelcevic
Databricks Employee
Databricks Employee

Hi @Dali1,

+1 to @pradeep_singh, on shared clusters, tasks inherit cluster-installed libraries, so you won’t get a clean, versioned environment. Use a job cluster (new_cluster) or switch to serverless jobs with an environment per task for isolation. With serverless, define job.environments and set environment_key on your wheel/script task. List your wheel/requirements in dependencies to pin exact version.