cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

How to make a DLT pipeline trigger another pipeline?

guangyi
New Contributor III

For example, I have 2 DLT pipelines, one is used for computing user gender distribution, another is used for compute user location distribution.

 

In the first pipeline, I follow the medallion architecture creating the bronze, silver, gold table one by one.

 

In the second pipeline, considering it related to the user data also, I want reuse the silver table created in the first pipeline.

 

I implemented the pipeline code successfully. But I don’t know how to achieve this: when the silver table data been updated by the first pipeline, the second pipeline should be triggered as well for updating the data immediately to reflect the latest distribution 



1 ACCEPTED SOLUTION

Accepted Solutions

Kaniz_Fatma
Community Manager
Community Manager

 

Hi @guangyi, To trigger your second pipeline when the silver table data is updated by the first pipeline, you can follow these steps:

  1. Event-Based Trigger:

  2. Azure Data Factory (ADF):

If you have any further questions or need additional guidance, feel free to ask! 😊

View solution in original post

1 REPLY 1

Kaniz_Fatma
Community Manager
Community Manager

 

Hi @guangyi, To trigger your second pipeline when the silver table data is updated by the first pipeline, you can follow these steps:

  1. Event-Based Trigger:

  2. Azure Data Factory (ADF):

If you have any further questions or need additional guidance, feel free to ask! 😊

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!