topic Re: Asset Bundles Email/Notifications Prod ONly in Data Engineering

Asset Bundles Email/Notifications Prod ONly

rriley2 — Wed, 19 Mar 2025 20:45:24 GMT

Howdy,

I've got a job 'job1' and my dev/stg/prod target in my databricks.yaml

Currently, I have this configurationfor my job:

email_notifications:
 on_success:
  - me@myorg.com
 on_failure:
  - me@myorg.com
webhook_notifications:
 on_failure:
  - id: ${var.webhook_uuid}

I'd like to ultimately like the following conditions:

dev: nothing, this is paused (solved this problem already)
stg: email only on failure
prod: email on success and failure, webhook on failure.

I've tried just setting the variable to be empty and this is a violation in the databricks validate task.

I've tried setting the webhook uuid to some junk and that also fails.

Any advice would be lovely!

Re: Asset Bundles Email/Notifications Prod ONly

Brahmareddy — Thu, 20 Mar 2025 03:30:33 GMT

Hi rriley2,

How are you doing today?, As per my understanding, Yeah, Databricks doesn’t allow empty values for email_notifications or webhook_notifications, so the trick is to use YAML conditionals and defaults to handle different environments without validation errors. A good way to do this is by setting up YAML anchors to define common notification settings and then overriding them per environment. For example, in dev, you can just set {} for both email and webhook to disable notifications. In stg, keep email only for failures, and in prod, enable both success/failure emails and webhook on failure. This avoids using junk values while keeping the config clean and flexible. Let me know if you need a sample YAML file—I’ve got you covered!

Regards,

Brahma

Re: Asset Bundles Email/Notifications Prod ONly

rriley2 — Thu, 20 Mar 2025 03:49:33 GMT

Hmmm so something like this:

targets:
  dev:
    resources:
      jobs:
        Workflow1:
          email_notifications: {}
          webhook_notifications: {}
  stage:
    resources:
      jobs:
        Workflow1:
          email_notifications:
            on_success:
              - me@myorg.com
            on_failure:
              - me@myorg.com
          webhook_notifications:
            on_failure:
              - id: stg_id
  prod:
    resources:
      jobs:
        Workflow1:
          email_notifications:
            on_success:
              - me@myorg.com
            on_failure:
              - me@myorg.com
          webhook_notifications:
            on_failure:
              - id: prod_id

So I would have to push the resource into the target keys to add environment variance. That makes sense. If I get this to work, I will post again.

Thanks!

Re: Asset Bundles Email/Notifications Prod ONly

rriley2 — Thu, 20 Mar 2025 18:34:38 GMT

This is exactly what I did! Thanks!