How to setup Job notifications using Microsoft Teams webhook ?

[ravi28]

Couple of things I tried:

1. I created a webhook connector in msft teams and copied it Notifications destinations via Admin page -> New destination -> from dropdown I selected Microsoft teams -> added webhook url and saved it.

outcome: I don't get the option to test it and the destination doesn't show up in databricks job notification settings

Second, instead of selecting the type Teams, while creating destination, I created the it by selecting webhook and copied msft teams webhook in the url field and saved it.

outcome: the test option failed with 400 error.

[Kaniz]

Hi @Ravi Sharvirala​, Here is an article for your query -

• https://learn.microsoft.com/en-us/azure/databricks/administration-guide/workspace/settings/notificat...

[ravi28]

@Kaniz Fatma​ thank you for answering.

The steps I tried were using the same article and It didn't help.

[Kaniz]

@Ravi Sharvirala​, If you received a 400 error when testing the destination with the Microsoft Teams webhook URL, double-check the URL for any potential issues, such as incorrect formatting or missing components. Ensure the URL is copied accurately from the Microsoft Teams webhook connector setup.

[ravi28]

@Kaniz Fatma​ I did. I used the same url to send a message using python requests and it worked.

I saved the webhook in two formats as Msft teams inbuilt webhook connector and as a Webhook.

Tried with both formats.

1. Msft teams inbuilt - I don't even see the test button
2. Webhook - I get an 400 error

[Kaniz]

Hi @Ravi Sharvirala​, Notification destinations currently have the following limitations:

• You can only configure notifications for Databricks SQL and Jobs.
• For jobs, only Slack and Webhook are presently supported as destinations.

[Anonymous]

Hi @Ravi Sharvirala​ 

Thank you for posting your question in our community! We are happy to assist you.

To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?

This will also help other community members who may have similar questions in the future. Thank you for your participation and let us know if you need any further assistance! 

[ravi28]

@Vidula Khanna​ It's not resolved or answered yet.

[Thijs]

MS Teams channel notifications are not supported for jobs, see the section "limitations" at the bottom of this page.

[AgileThought]

La respuesta rápida es: "Programáticamente". Revisa el artículo 

Send a Message to Microsoft Teams via Databricks

 https://medium.com/analytics-vidhya/send-a-message-to-microsoft-teams-via-databricks-84dd8ca06dc7 

La implementación de Azure Databricks para añadir destinos de notificación directamente como canales de Teams aún se encuentra en Preview y efectivamente aún no es funcional. No solo no muestra el enlace para probar la conexión, ni siquiera lo presenta listado al momento de intentar agregar el destino de notificación al job.

Por otra parte, haciendo uso de la app Incoming Webhook en el Teams, se puede configurar exitosamente el destino de notificación como Webhook, pero al probar el mismo notarás que obtienes un error 400, el cual se deriva del formato como esta construido el mensaje.

De manera que, a falta de una app que permita al Ms Team utilizar una opción más genérica de mensaje. La alternativa más viable es realizar la construcción y envío del mensaje de manera programática a fin de asegurar que el formato sea el requerido. 

En lo personal, me pareció más viable hacer a un lado el Ms Team y conectar Slack. 

[youssefmrini]

You can set up job notifications for Databricks jobs using Microsoft Teams webhooks by following these steps:

1. Set up a Microsoft Teams webhook:

• Go to the channel where you want to receive notifications in Microsoft Teams.
• Click on the "..." icon next to the channel name and select "Connectors".
• Search for "Webhook" and select the "Incoming Webhook" connector.
• Provide a name for the webhook and an optional image, then click "Create".
• Copy the webhook URL to use later in the Databricks job configuration.

1. Set up a Databricks secret scope:

• In the Databricks workspace, go to "Secrets" under "User Settings".
• Create a new secret scope and give it a name, such as "teams-notification".
• Add a secret for the webhook URL that you copied in the previous step.

1. Configure the Databricks job:

• In the Databricks workspace, open the job that you want to configure notifications for.

• Under "Advanced Options", click on "Edit" next to "Output".

• Add the following configuration options for the job:

  text

  "notifications": {
    "onSuccess": [
      {
        "type": "Webhook",
        "options": {
          "url": "/secrets/vault-uri/teams-notification/webhook-url",
          "payload": {
            "text": "The job {{job_name}} completed successfully."
          }
        }
      }
    ],
    "onFailure": [
      {
        "type": "Webhook",
        "options": {
          "url": "/secrets/vault-uri/teams-notification/webhook-url",
          "payload": {
            "text": "The job {{job_name}} failed with error: \n{{error_message}}"
          }
        }
      }
    ]
  }
  • Replace /secrets/vault-uri with the URI for your secret scope.
  • Replace webhook-url with the name of the secret you created for the webhook URL.
  • Customize the message text as needed, using placeholder variables such as {{job_name}} and {{error_message}}.

1. Save and run the job:

• Save the job configuration and run the job as usual.
• When the job completes, you should receive a notification in the Microsoft Teams channel you configured.

And that's it! You have now configured job notifications using Microsoft Teams webhooks in Databricks.