run a Databricks notebook on serverless environment version 4 with Asset Bundles

[DatabricksEngi1]

Hi everyone,
I’m working with Databricks Asset Bundles and running jobs that use notebooks (.ipynb).

According to the documentation, it should be possible to set an environment version for serverless jobs. I want to force all of my notebook tasks to run with environment version 4.

• When I omit the cluster definition (to rely on serverless), the job always defaults to environment version 1.
• I don’t select the environment through the UI, since I’m managing everything through DAB YAML.
• Adding an environment_key works for wheel tasks, but not for notebook tasks.

How can I make a .ipynb notebook job run on serverless with environment version 4 using Asset Bundles?
Is there a way to enforce this, or are notebook tasks limited to environment v1 only?

Thanks in advance for any insights.

[szymon_dybczak]

Hi @DatabricksEngi1 

When you're defining job in DAB you're using job mapping. One of the key of job mapping is called environments. This is the one you're looking for:

Databricks Asset Bundles resources - Azure Databricks | Microsoft Learn

[DatabricksEngi1]

Thank you for your response, but according to the documentation, this does not work with .ipynb files.
My jobs use this extension - would it be possible?

[szymon_dybczak]

Hi @DatabricksEngi1 ,

ipynb is a default format for notebooks. Could you provide me a documentation entry where they're saying it's not supported? In my opinion - it should work.

[DatabricksEngi1]

when I try to configure it in the yaml :
resources:
  jobs:
    ofirtest:
      name: ofirtest
      tasks:
        - task_key: ofirtest
          notebook_task:
            notebook_path: /Workspace/Users/username/Untitled Notebook
              2025-09-15 14:22:51

            source: WORKSPACE
          environment_key: default
      queue:
        enabled: true
      environments:
        - environment_key: default
          spec:
            environment_version: "4"
      performance_target: PERFORMANCE_OPTIMIZED  

this is how it looks like:

[szymon_dybczak]

Thanks for sharing @DatabricksEngi1. So it seems that currently you can't set this programatically with DAB. I've found official answer of databricks employee to similiar question as your. So unfrotunately, currently serverless environment version must be set at the notebook level through UI

Solved: Programatic selection of serverless compute for no... - Databricks Community - 110225

[DatabricksEngi1]

Thank you very much for your comments.

Hope there will be a solution soon.

[szymon_dybczak]

Yes, I think this is something that will be added in the future 🙂