Hello,

I have an ETL pipeline in Databricks that works perfectly when I execute it manually in the notebook using an all-purpose cluster. However, when I try to schedule it using a job cluster, it fails immediately with the error message: 'Azure container does not exist.'

I attempted to run the job using the same all-purpose cluster, but encountered the same error. Even a test job with a simple print command resulted in the same error. It seems I'm unable to execute job runs from this Databricks workspace.

Does anyone have any insights on what might be causing this?

Thank you very much in advance