- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-16-2026 06:27 AM - edited 02-16-2026 06:28 AM
Does anyone know of a way to see what Lakeflow pipelines are using the same gateway? We have a gateway connected to a SQL Server that serves multiple individual pipelines but I cannot find a way to see what those are. I've tried system tables. Any insight would be appreciated.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-16-2026 08:54 AM
Hi @pdiamond,
You can use REST API or databricks cli (which under the hood make REST API calls for you anyway).
Here's an endpoint you're looking for:
Get a pipeline | Pipelines API | REST API reference | Databricks on AWS
Then in payload look for spec object:
Inside that object you should find another called ingestion_definiton which contains attribute ingestion_gateway_id:
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-16-2026 09:48 AM
Thanks @szymon_dybczak - this worked perfectly. This is what I ended up throwing together to see what I was looking for:
from databricks.sdk import WorkspaceClient
url = f"{DATABRICKS_HOST}/api/2.0/pipelines"
headers = wc.config.authenticate()
wc = WorkspaceClient()
DATABRICKS_HOST = wc.config.host
import requests
response = requests.get(url, headers=headers)
payload = response.json()
pipeline_ids = [item.get("pipeline_id") for item in payload.get("statuses", []) if "pipeline_id" in item]
pipeline_payloads = []
for pid in pipeline_ids:
detail_url = f"{DATABRICKS_HOST}/api/2.0/pipelines/{pid}"
detail_response = requests.get(detail_url, headers=headers)
detail_payload = detail_response.json()
pipeline_payloads.append(detail_payload)
from pyspark.sql import Row
rows = [
Row(
pipeline_id=payload.get("pipeline_id"),
name=payload.get("spec", {}).get("name"),
ingestion_gateway_id=payload.get("spec", {}).get("ingestion_definition", {}).get("ingestion_gateway_id")
)
for payload in pipeline_payloads
]
df_pipelines = spark.createDataFrame(rows)
display(df_pipelines)