Databricks Job Failure + Service now Integration

[Phani1]

Hi Team,

Could you please suggest how to raise the service now ticket, in case of Databricks job failure?

Regards ,

Phanindra

[Phani1]

Sorry, my question might be confusing,

My question is for example I have one job which loads data / performs some transformation on delta tables, job got failed due to some issues, In databricks we have job failure notification but we want to integrate with service now.

We would like to raise a ticket in the service now system saying the job failed due to some reason  and support engineer will work on the ticket and will analyze the reasons for failing. 

Basically we would like to integrate with Databricks and service now.

[Swastik_Mishra]

Hi @Phani1, To integrate Databricks job failure notifications with ServiceNow, you can use the webhook method. This allows Databricks to send an HTTP POST request (webhook) to a designated endpoint in ServiceNow whenever a job fails. By doing so, you can automatically raise a ticket in the ServiceNow system, notifying the support engineers about the job failure, and enabling them to analyze the reasons for the failure.

• Create a Custom Endpoint in ServiceNow:

1. Create a new "Inbound REST Web Service" or "Scripted REST API" and define the endpoint URL where Databricks will send the webhook notification.

• Configure Databricks Job Failure Notification:

1. In Databricks, go to the job that you want to monitor for failures.
2. Click on Edit Notifications to configure the job
3. Click on Add Notification, and in Select a system destination, select a destination, and click the check box for failure job notification type to send to that destination.
4. If it’s showing  Add new system destination then click on that it will open a new tab,
5. In that click Add Destination select Select the "Webhook" option and provide the URL of the custom endpoint you created in ServiceNow.

For more information about webhook you can refer to the below documentation links

https://docs.databricks.com/workflows/jobs/job-notifications.html

https://docs.databricks.com/administration-guide/workspace/settings/notification-destinations.html#a...

[Swastik_Mishra]

Hi @Phani1, You can use the webhook method to integrate Databricks job failure notifications with ServiceNow. This allows Databricks to send an HTTP POST request (webhook) to a designated endpoint in ServiceNow whenever a job fails. By doing so, you can automatically raise a ticket in the ServiceNow system, notifying the support engineers about the job failure, and enabling them to analyze the reasons for the failure.

• Create a Custom Endpoint in ServiceNow:

1. Create a new "Inbound REST Web Service" or "Scripted REST API" and define the endpoint URL where Databricks will send the webhook notification.

• Configure Databricks Job Failure Notification:

1. In Databricks, go to the job that you want to monitor for failures.
2. Click on Edit Notifications to configure the job
3. Click on Add Notification, and in Select a system destination, select a destination, and click the check box for failure job notification type to send to that destination.
4. If it’s showing  Add new system destination then click on that it will open a new tab,
5. In that click Add Destination select Select the "Webhook" option and provide the URL of the custom endpoint you created in ServiceNow.

For more information about webhook you can refer to the below documentation links

https://docs.databricks.com/workflows/jobs/job-notifications.html

https://docs.databricks.com/administration-guide/workspace/settings/notification-destinations.html#a...