Conditional Logic in Databricks Asset Bundles using Go Templates

[Seunghyun]

I am defining a job using Databricks Asset Bundles (DABs) as follows:

YAML

resources:
  jobs:
    job_name:
      ...
      schedule:
        {{ if eq ${var.env} "prd" }}
        pause_status: "UNPAUSED"
        {{ else }}
        pause_status: "PAUSED"
        {{ end }}

I understand that Databricks bundle resource YAML files support Go-based templating. My goal was to deploy dynamic bundle resources based on the value of a bundle variable.

In the example above, I wanted to set the pause_status to "UNPAUSED" if the var.env value is "prd", and "PAUSED" for other development environments.

However, when I run databricks bundle validate, I encounter an error: did not find expected key. It seems that the template is unable to correctly reference the bundle variable, or the syntax is breaking the YAML structure.

As a side note, I have already confirmed that adding a pause_status variable per target in the Databricks configuration YAML works fine. I was simply curious if this alternative template-based approach is possible.

Has anyone successfully implemented this or found a workaround?

[pradeep_singh]

Intresting . I havent seen this go style templating with DABS . Typically you would define your variables in databricks.yml and then override the variable per environment . 

# databricks.yml

variables:
  job_pause_status:
    description: Pause/unpause job schedule by environment
    default: PAUSED   # sensible default

resources:
  jobs:
    job_name:
      name: job_name
      schedule:
        quartz_cron_expression: "0 0 8 * * ?"
        timezone_id: "UTC"
        pause_status: ${var.job_pause_status}

Override per target 

targets:
  dev:
    mode: development
    default: true
    variables:
      job_pause_status: PAUSED

  qa:
    mode: development
    variables:
      job_pause_status: PAUSED

  prd:
    mode: production
    variables:
      job_pause_status: UNPAUSED

[saurabh18cs · Accepted solution]

also check this : https://community.databricks.com/t5/data-engineering/if-else-conditions-in-databricks-asset-bundles/...

[SteveOstrowski · Accepted solution]

Hi @Seunghyun,

Go template syntax ({{if}}, {{eq}}, etc.) is only supported in bundle project templates, which are the .tmpl files used during "databricks bundle init" to scaffold new projects. It is not supported inside your regular databricks.yml configuration file, which is why you are seeing the "did not find expected key" validation error.

There are several approaches to achieve per-environment conditional behavior for settings like pause_status without Go templates:

APPROACH 1: TARGET-SPECIFIC OVERRIDES (RECOMMENDED)

You can override resource settings directly within each target definition. Settings defined under a specific target merge with and override the top-level resource definitions:

resources:
jobs:
  job_name:
    schedule:
      quartz_cron_expression: "0 0 8 * * ?"
      timezone_id: "UTC"
      pause_status: "PAUSED"

targets:
dev:
  mode: development
prd:
  resources:
    jobs:
      job_name:
        schedule:
          pause_status: "UNPAUSED"

In this setup, the top-level definition defaults pause_status to PAUSED, and the prd target overrides it to UNPAUSED. The dev target inherits the default PAUSED value.

APPROACH 2: USING CUSTOM VARIABLES

You can define a custom variable and set its value per target:

variables:
pause_status:
  description: "Whether job schedules are paused"
  default: "PAUSED"

resources:
jobs:
  job_name:
    schedule:
      quartz_cron_expression: "0 0 8 * * ?"
      timezone_id: "UTC"
      pause_status: ${var.pause_status}

targets:
dev:
  mode: development
  variables:
    pause_status: "PAUSED"
prd:
  mode: production
  variables:
    pause_status: "UNPAUSED"

This is essentially what you mentioned is already working for you. This is a clean, idiomatic pattern and is the recommended way to parameterize values across targets.

APPROACH 3: USING PRESETS (trigger_pause_status)

If you want to control pause_status for all jobs and triggers at the target level without touching individual resource definitions, you can use the trigger_pause_status preset:

targets:
dev:
  mode: development
  presets:
    trigger_pause_status: PAUSED
prd:
  mode: production
  presets:
    trigger_pause_status: UNPAUSED

Note that development mode already pauses all schedules and triggers by default, so setting PAUSED explicitly in dev is optional. The presets approach is convenient when you have many jobs and want a blanket policy per environment.

A NOTE ABOUT DEVELOPMENT MODE

When you set mode: development on a target, Databricks Asset Bundles automatically pauses all schedules and triggers on deployed resources. So if your primary goal is simply to have schedules paused in dev and running in prod, using mode: development for dev and mode: production for prod may be all you need.

REFERENCES

- Databricks Asset Bundles variables: https://docs.databricks.com/en/dev-tools/bundles/variables.html
- Bundle deployment modes: https://docs.databricks.com/en/dev-tools/bundles/deployment-modes.html
- Bundle configuration reference: https://docs.databricks.com/en/dev-tools/bundles/reference.html
- Bundle templates (where Go template syntax is supported): https://docs.databricks.com/en/dev-tools/bundles/templates.html

* This reply used an agent system I built to research and draft this response based on the wide set of documentation I have available and previous memory. I personally review the draft for any obvious issues and for monitoring system reliability and update it when I detect any drift, but there is still a small chance that something is inaccurate, especially if you are experimenting with brand new features.

If this answer resolves your question, could you mark it as "Accept as Solution"? That helps other users quickly find the correct fix.