maikel
Contributor III

Hello @Ashwin_DSA ,

I am trying to implement environment distinction for notebooks and have some questions 🙂
My resources:

resources/
  ├── jobs/
  │   ├── notebook_a.yml
  │   ├── notebook_b.yml
  │   ├── notebook_c.yml          
  │   ├── notebook_d.yml
  ├── debug/
      ├── notebook_debug.yml

And my databricks.yml:

bundle:
name: example_bundle

include:
- resources/jobs/*.yml

targets:
local:
mode: development
default: true
workspace:
host: https://dev-env.url.databricks.com

dev:
mode: production
default: false
workspace:
host: https://dev-env.url.databricks.com
permissions:
- ...

preprod:
mode: production
default: false
workspace:
host: https://preprod-env.url.databricks.com


Example notebook_a.yml:

resources:
jobs:
notebook_a:
name: notebook_a_${bundle.target}
description: some example notebook

environments:
- environment_key: default
spec:
client: "4"
dependencies:
- ../../dist/*.whl

tasks:
- task_key: notebook_a
environment_key: default
notebook_task:
notebook_path: ${workspace.file_path}/src/notebooks/notebook_a.yml
source: WORKSPACE
base_parameters:
example_parameter: "placeholder"

max_concurrent_runs: 1

tags:
...

permissions:
...

I would like to have debug/notebook_debug deployed only to local and dev. Is there a possibility to do so in a shape of databricks.yml file which I have currently without a need to define a whole resource under the specific targets? 

I would like to achieve something like this:

bundle:
  name: example_bundle

include:
  - resources/jobs/*.yml

targets:
  local:
    mode: development
    default: true
    workspace:
      host: https://dev-env.url.databricks.com
    include:
      - resources/debug/*.yml

  dev:
    mode: production
    default: false
    workspace:
      host: https://dev-env.url.databricks.com
    permissions:
      - ...
    include:
      - resources/debug/*.yml

  preprod:
    mode: production
    default: false
    workspace:
      host: https://preprod-env.url.databricks.com

Thanks a lot for the support!