szymon_dybczak
Esteemed Contributor III

Hi @jeremy98 ,

Maybe you can use Jobs API 2.2 for that scenario and based on returned payload orchestrate your pipeline?

Updating from Jobs API 2.1 to 2.2 | Databricks Documentation