topic Re: How to setup Job notifications using Microsoft Teams webhook ? in Data Engineering

How to setup Job notifications using Microsoft Teams webhook ?

ravi28 — Thu, 08 Jun 2023 00:40:47 GMT

Couple of things I tried:

1. I created a webhook connector in msft teams and copied it Notifications destinations via Admin page -> New destination -> from dropdown I selected Microsoft teams -> added webhook url and saved it.

outcome: I don't get the option to test it and the destination doesn't show up in databricks job notification settings

Second, instead of selecting the type Teams, while creating destination, I created the it by selecting webhook and copied msft teams webhook in the url field and saved it.

outcome: the test option failed with 400 error.

Re: How to setup Job notifications using Microsoft Teams webhook ?

ravi28 — Mon, 12 Jun 2023 13:52:21 GMT

@Kaniz Fatma thank you for answering.

The steps I tried were using the same article and It didn't help.

Re: How to setup Job notifications using Microsoft Teams webhook ?

ravi28 — Mon, 12 Jun 2023 14:26:31 GMT

@Kaniz Fatma I did. I used the same url to send a message using python requests and it worked.

I saved the webhook in two formats as Msft teams inbuilt webhook connector and as a Webhook.

Tried with both formats.

Msft teams inbuilt - I don't even see the test button
Webhook - I get an 400 error

Re: How to setup Job notifications using Microsoft Teams webhook ?

Anonymous — Wed, 14 Jun 2023 06:37:40 GMT

Hi @Ravi Sharvirala

Thank you for posting your question in our community! We are happy to assist you.

To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?

This will also help other community members who may have similar questions in the future. Thank you for your participation and let us know if you need any further assistance!

Re: How to setup Job notifications using Microsoft Teams webhook ?

ravi28 — Wed, 14 Jun 2023 13:30:51 GMT

@Vidula Khanna It's not resolved or answered yet.

Re: How to setup Job notifications using Microsoft Teams webhook ?

Thijs — Wed, 21 Jun 2023 20:07:31 GMT

MS Teams channel notifications are not supported for jobs, see the section "limitations" at the bottom of this page.

Re: How to setup Job notifications using Microsoft Teams webhook ?

AgileThought — Mon, 07 Aug 2023 14:43:28 GMT

La respuesta rápida es: "Programáticamente". Revisa el artículo

Send a Message to Microsoft Teams via Databricks

https://medium.com/analytics-vidhya/send-a-message-to-microsoft-teams-via-databricks-84dd8ca06dc7

La implementación de Azure Databricks para añadir destinos de notificación directamente como canales de Teams aún se encuentra en Preview y efectivamente aún no es funcional. No solo no muestra el enlace para probar la conexión, ni siquiera lo presenta listado al momento de intentar agregar el destino de notificación al job.

Por otra parte, haciendo uso de la app Incoming Webhook en el Teams, se puede configurar exitosamente el destino de notificación como Webhook, pero al probar el mismo notarás que obtienes un error 400, el cual se deriva del formato como esta construido el mensaje.

De manera que, a falta de una app que permita al Ms Team utilizar una opción más genérica de mensaje. La alternativa más viable es realizar la construcción y envío del mensaje de manera programática a fin de asegurar que el formato sea el requerido.

En lo personal, me pareció más viable hacer a un lado el Ms Team y conectar Slack.

Re: How to setup Job notifications using Microsoft Teams webhook ?

youssefmrini — Tue, 08 Aug 2023 15:49:56 GMT

You can set up job notifications for Databricks jobs using Microsoft Teams webhooks by following these steps:

Set up a Microsoft Teams webhook:

Go to the channel where you want to receive notifications in Microsoft Teams.
Click on the "..." icon next to the channel name and select "Connectors".
Search for "Webhook" and select the "Incoming Webhook" connector.
Provide a name for the webhook and an optional image, then click "Create".
Copy the webhook URL to use later in the Databricks job configuration.

Set up a Databricks secret scope:

In the Databricks workspace, go to "Secrets" under "User Settings".
Create a new secret scope and give it a name, such as "teams-notification".
Add a secret for the webhook URL that you copied in the previous step.

Configure the Databricks job:

In the Databricks workspace, open the job that you want to configure notifications for.
Under "Advanced Options", click on "Edit" next to "Output".
Add the following configuration options for the job:
text
"notifications": {
  "onSuccess": [
    {
      "type": "Webhook",
      "options": {
        "url": "/secrets/vault-uri/teams-notification/webhook-url",
        "payload": {
          "text": "The job {{job_name}} completed successfully."
        }
      }
    }
  ],
  "onFailure": [
    {
      "type": "Webhook",
      "options": {
        "url": "/secrets/vault-uri/teams-notification/webhook-url",
        "payload": {
          "text": "The job {{job_name}} failed with error: \n{{error_message}}"
        }
      }
    }
  ]
}
- Replace /secrets/vault-uri with the URI for your secret scope.
- Replace webhook-url with the name of the secret you created for the webhook URL.
- Customize the message text as needed, using placeholder variables such as {{job_name}} and {{error_message}}.

Save and run the job:

Save the job configuration and run the job as usual.
When the job completes, you should receive a notification in the Microsoft Teams channel you configured.

And that's it! You have now configured job notifications using Microsoft Teams webhooks in Databricks.