Hi @srikanthp24,
Yeah... this is something others have run into as well.
When you create a Lakeflow Connect ingestion pipeline via the UI wizard, the "Schedules and notifications" step (step 6) only configures alerts for that pipeline. The gateway pipeline gets created alongside it, but the wizard does not surface a notification option for it. The good news is that the gateway is just a regular Lakeflow pipeline under the hood, so you can absolutely set up failure alerts on it. You just need to do it as a separate step after the wizard finishes.
The easiest option is to add notifications through the pipeline UI. Once your pipeline is created, go to Jobs & Pipelines in the sidebar and find your gateway pipeline (it will be named the one you entered in the "Gateway" field during setup). Open it, click into the pipeline settings, and you will see a Notifications section. Click "Add notification", enter your email address, and select the failure events you care about (such as "on update failure" or "on fatal failure"). Save, and you are done. The docs walk through this here: Add email notifications for pipeline events.
If you prefer a programmatic approach, just send a PUT request to /api/2.0/pipelines/{gateway_pipeline_id} with a body like this:
{
"notifications": [
{
"email_recipients": ["your-email@example.com"],
"alerts": [
"on-update-failure",
"on-update-fatal-failure",
"on-flow-failure"
]
}
]
}
Or with the Databricks CLI:
databricks pipelines update --pipeline-id <gateway_pipeline_id> --json '{
"notifications": [
{
"email_recipients": ["your-email@example.com"],
"alerts": ["on-update-failure", "on-update-fatal-failure", "on-flow-failure"]
}
]
}'
You can grab the gateway pipeline ID from the pipeline details page in the UI. More details on this are in the Common pipeline maintenance tasks doc.
If you manage pipelines with DABs, you can define notifications directly in the gateway pipeline's YAML definition so they are version-controlled from the start:
pipelines:
gw_pipeline:
name: 'my-gateway'
continuous: true
notifications:
- email_recipients:
- "team@example.com"
alerts:
- "on-update-failure"
- "on-update-fatal-failure"
- "on-flow-failure"
gateway_definition:
connection_name: ${var.connection_name}
gateway_storage_catalog: ${var.staging_catalog}
gateway_storage_schema: ${var.staging_schema}
The SQL Server pipeline bundle examples in the docs actually show this as a commented-out option on the ingestion pipeline. The same block works on the gateway pipeline definition too.
As a starting point, navigate to your gateway pipeline in Jobs & Pipelines, open its settings, and add your email notification there. It takes about 30 seconds and covers most use cases. If you want something more automated or integrated into your deployment workflow, go with the API or DABs approach.
Hope this helps. Let us know if you run into any issues getting it set up.
If this answer resolves your question, could you mark it as โAccept as Solutionโ? That helps other users quickly find the correct fix.
Regards,
Ashwin | Delivery Solution Architect @ Databricks
Helping you build and scale the Data Intelligence Platform.
***Opinions are my own***
