cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

How to setup Job notifications using Microsoft Teams webhook ?

ravi28
New Contributor III

Couple of things I tried:

1. I created a webhook connector in msft teams and copied it Notifications destinations via Admin page -> New destination -> from dropdown I selected Microsoft teams -> added webhook url and saved it.

outcome: I don't get the option to test it and the destination doesn't show up in databricks job notification settings

Second, instead of selecting the type Teams, while creating destination, I created the it by selecting webhook and copied msft teams webhook in the url field and saved it.

outcome: the test option failed with 400 error.

10 REPLIES 10

Kaniz
Community Manager
Community Manager

ravi28
New Contributor III

@Kaniz Fatma​ thank you for answering.

The steps I tried were using the same article and It didn't help.

Kaniz
Community Manager
Community Manager

@Ravi Sharvirala​, If you received a 400 error when testing the destination with the Microsoft Teams webhook URL, double-check the URL for any potential issues, such as incorrect formatting or missing components. Ensure the URL is copied accurately from the Microsoft Teams webhook connector setup.

ravi28
New Contributor III

@Kaniz Fatma​ I did. I used the same url to send a message using python requests and it worked.

I saved the webhook in two formats as Msft teams inbuilt webhook connector and as a Webhook.

imageTried with both formats.

  1. Msft teams inbuilt - I don't even see the test button
  2. Webhook - I get an 400 error

Kaniz
Community Manager
Community Manager

Hi @Ravi Sharvirala​, Notification destinations currently have the following limitations:

  • You can only configure notifications for Databricks SQL and Jobs.
  • For jobs, only Slack and Webhook are presently supported as destinations.

Anonymous
Not applicable

Hi @Ravi Sharvirala​ 

Thank you for posting your question in our community! We are happy to assist you.

To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?

This will also help other community members who may have similar questions in the future. Thank you for your participation and let us know if you need any further assistance! 

ravi28
New Contributor III

@Vidula Khanna​ It's not resolved or answered yet.

Thijs
New Contributor III

MS Teams channel notifications are not supported for jobs, see the section "limitations" at the bottom of this page.

AgileThought
New Contributor II

La respuesta rápida es: "Programáticamente". Revisa el artículo 

Send a Message to Microsoft Teams via Databricks

 https://medium.com/analytics-vidhya/send-a-message-to-microsoft-teams-via-databricks-84dd8ca06dc7 

La implementación de Azure Databricks para añadir destinos de notificación directamente como canales de Teams aún se encuentra en Preview y efectivamente aún no es funcional. No solo no muestra el enlace para probar la conexión, ni siquiera lo presenta listado al momento de intentar agregar el destino de notificación al job.

Por otra parte, haciendo uso de la app Incoming Webhook en el Teams, se puede configurar exitosamente el destino de notificación como Webhook, pero al probar el mismo notarás que obtienes un error 400, el cual se deriva del formato como esta construido el mensaje.

De manera que, a falta de una app que permita al Ms Team utilizar una opción más genérica de mensaje. La alternativa más viable es realizar la construcción y envío del mensaje de manera programática a fin de asegurar que el formato sea el requerido. 

En lo personal, me pareció más viable hacer a un lado el Ms Team y conectar Slack. 

youssefmrini
Honored Contributor III
Honored Contributor III

You can set up job notifications for Databricks jobs using Microsoft Teams webhooks by following these steps:

  1. Set up a Microsoft Teams webhook:
  • Go to the channel where you want to receive notifications in Microsoft Teams.
  • Click on the "..." icon next to the channel name and select "Connectors".
  • Search for "Webhook" and select the "Incoming Webhook" connector.
  • Provide a name for the webhook and an optional image, then click "Create".
  • Copy the webhook URL to use later in the Databricks job configuration.
  1. Set up a Databricks secret scope:
  • In the Databricks workspace, go to "Secrets" under "User Settings".
  • Create a new secret scope and give it a name, such as "teams-notification".
  • Add a secret for the webhook URL that you copied in the previous step.
  1. Configure the Databricks job:
  • In the Databricks workspace, open the job that you want to configure notifications for.

  • Under "Advanced Options", click on "Edit" next to "Output".

  • Add the following configuration options for the job:

    text
    "notifications": {
      "onSuccess": [
        {
          "type": "Webhook",
          "options": {
            "url": "/secrets/vault-uri/teams-notification/webhook-url",
            "payload": {
              "text": "The job {{job_name}} completed successfully."
            }
          }
        }
      ],
      "onFailure": [
        {
          "type": "Webhook",
          "options": {
            "url": "/secrets/vault-uri/teams-notification/webhook-url",
            "payload": {
              "text": "The job {{job_name}} failed with error: \n{{error_message}}"
            }
          }
        }
      ]
    }
    • Replace /secrets/vault-uri with the URI for your secret scope.
    • Replace webhook-url with the name of the secret you created for the webhook URL.
    • Customize the message text as needed, using placeholder variables such as {{job_name}} and {{error_message}}.
  1. Save and run the job:
  • Save the job configuration and run the job as usual.
  • When the job completes, you should receive a notification in the Microsoft Teams channel you configured.

And that's it! You have now configured job notifications using Microsoft Teams webhooks in Databricks.

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!