emma_s
Databricks Employee
Databricks Employee

Hi, 

As you've correctly identified workspace base environments aren't currently supported by DABs as they're a relatively new feature. They are more meant to give a quick base environment to workspace users rather than used to deploy notebooks as jobs through DABS.

The correct way to do this is to specify the environment in the DAB and then use this environment for the job, rather than trying to use the workspace base environment. See the snippet below:

resources:
  jobs:
    serverless_job_environment:
      name: serverless_job_environment
      environments:
        - environment_key: default
          spec:
            environment_version: "2"
            dependencies:
              - "my-private-lib==1.2.3"
              - "requests==2.32.*"

      tasks:
        - task_key: nb_task
          notebook_task:
            notebook_path: ../src/notebook.ipynb
          environment_key: default

Docs for this here https://docs.databricks.com/aws/en/dev-tools/bundles/examples

 

I hope this helps.


Thanks,

Emma

 

View solution in original post