topic Re: Data Bricks workflow - Executing the workflow with different input parameters with concurrently in Data Engineering

Data Bricks workflow - Executing the workflow with different input parameters with concurrently

rameshybr — Tue, 13 Aug 2024 12:39:26 GMT

How can I trigger a workflow concurrently (multiple times) with different input parameters? Please share your thoughts or any related articles.

Re: Data Bricks workflow - Executing the workflow with different input parameters with concurrently

Doug-Leal — Tue, 13 Aug 2024 19:20:22 GMT

See tutorials/articles below with all the steps to create a workflow and pass parameters to the job/workflow:

Also, under the "Advanced Settings" when editing the Workflow, you have options to set the concurrency based on your requirements. Set this value higher than the default of 1 if you want to be able to perform multiple runs of the same job concurrently, but with different parameters.

Hope this helps!

-All the Best!

Doug

Re: Data Bricks workflow - Executing the workflow with different input parameters with concurrently

Retired_mod — Wed, 14 Aug 2024 08:01:29 GMT

Hi @rameshybr, Thanks for reaching out! Please review the response and let us know if it answers your question. Your feedback is valuable to us and the community.

If the response resolves your issue, kindly mark it as the accepted solution. This will help close the thread and assist others with similar queries.

We appreciate your participation and are here if you need further assistance!