https://community.databricks.com/t5/data-engineering/dlt-get-pipeline-id-and-update-id/m-p/82291#M36599 <DIV class=""><P>I need to insert pipeline_id and update_id in my Delta Live Table (DLT), the point being to know which pipeline created which row. How can I obtain this information?</P><P>I know you can get job_id and run_id from widgets but I don't know if these are the same things as pipeline_id and update_id.</P><P>I have a solution with event_hooks using Python but it looks wrong, I do not think it makes sense to listen for events in my DLT pipeline for this purpose.</P><P>#dlt</P></DIV> Thu, 08 Aug 2024 00:34:34 GMT Zeruno 2024-08-08T00:34:34Z https://community.databricks.com/t5/data-engineering/dlt-get-pipeline-id-and-update-id/m-p/82291#M36599 <DIV class=""><P>I need to insert pipeline_id and update_id in my Delta Live Table (DLT), the point being to know which pipeline created which row. How can I obtain this information?</P><P>I know you can get job_id and run_id from widgets but I don't know if these are the same things as pipeline_id and update_id.</P><P>I have a solution with event_hooks using Python but it looks wrong, I do not think it makes sense to listen for events in my DLT pipeline for this purpose.</P><P>#dlt</P></DIV> Thu, 08 Aug 2024 00:34:34 GMT https://community.databricks.com/t5/data-engineering/dlt-get-pipeline-id-and-update-id/m-p/82291#M36599 Zeruno 2024-08-08T00:34:34Z https://community.databricks.com/t5/data-engineering/dlt-get-pipeline-id-and-update-id/m-p/82320#M36608 <P>Hi&nbsp;<a href="https://community.databricks.com/t5/user/viewprofilepage/user-id/115254">@Zeruno</a>&nbsp;,<BR /><BR />Those values are rather static. Maybe you can design process that as a first step will extract information from List Pipepline API and save them in delta table.<BR /><A href="https://docs.databricks.com/api/workspace/pipelines/listpipelines" target="_blank">List pipelines | Pipelines API | REST API reference | Databricks on AWS</A><BR />Than in your processing you lookup all needed information in this table.</P> Thu, 08 Aug 2024 07:16:38 GMT https://community.databricks.com/t5/data-engineering/dlt-get-pipeline-id-and-update-id/m-p/82320#M36608 szymon_dybczak 2024-08-08T07:16:38Z