You can set up job notifications for Databricks jobs using Microsoft Teams webhooks by following these steps:

1. Set up a Microsoft Teams webhook:

• Go to the channel where you want to receive notifications in Microsoft Teams.
• Click on the "..." icon next to the channel name and select "Connectors".
• Search for "Webhook" and select the "Incoming Webhook" connector.
• Provide a name for the webhook and an optional image, then click "Create".
• Copy the webhook URL to use later in the Databricks job configuration.

1. Set up a Databricks secret scope:

• In the Databricks workspace, go to "Secrets" under "User Settings".
• Create a new secret scope and give it a name, such as "teams-notification".
• Add a secret for the webhook URL that you copied in the previous step.

1. Configure the Databricks job:

• In the Databricks workspace, open the job that you want to configure notifications for.

• Under "Advanced Options", click on "Edit" next to "Output".

• Add the following configuration options for the job:

  text

  "notifications": {
    "onSuccess": [
      {
        "type": "Webhook",
        "options": {
          "url": "/secrets/vault-uri/teams-notification/webhook-url",
          "payload": {
            "text": "The job {{job_name}} completed successfully."
          }
        }
      }
    ],
    "onFailure": [
      {
        "type": "Webhook",
        "options": {
          "url": "/secrets/vault-uri/teams-notification/webhook-url",
          "payload": {
            "text": "The job {{job_name}} failed with error: \n{{error_message}}"
          }
        }
      }
    ]
  }
  • Replace /secrets/vault-uri with the URI for your secret scope.
  • Replace webhook-url with the name of the secret you created for the webhook URL.
  • Customize the message text as needed, using placeholder variables such as {{job_name}} and {{error_message}}.

1. Save and run the job:

• Save the job configuration and run the job as usual.
• When the job completes, you should receive a notification in the Microsoft Teams channel you configured.

And that's it! You have now configured job notifications using Microsoft Teams webhooks in Databricks.