Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
i want to fetch job id of child job which executes its own workflow , into parent job which executes its own workflow but depends on job child jobs job id?
looks like a common fan-out orchestration patter. your parent job should triger child job programatically and then capture maintain its run_id. you can use databricks jobs api endpoints to start child job.
Use POST /api/2.1/jobs/run-now or POST /api/2.1/jobs/runs/submit.