Data Bricks workflow - Executing the workflow with different input parameters with concurrently

[rameshybr]

How can I trigger a workflow concurrently (multiple times) with different input parameters? Please share your thoughts or any related articles.

[Doug-Leal · Accepted solution]

See tutorials/articles below with all the steps to create a workflow and pass parameters to the job/workflow: 

• https://docs.databricks.com/en/jobs/create-run-jobs.html 
• https://docs.databricks.com/en/jobs/parameter-value-references.html
• https://docs.databricks.com/en/jobs/create-run-jobs.html#job-run-with-different-params
• https://docs.databricks.com/en/jobs/create-run-jobs.html#task-parameters

Also, under the "Advanced Settings" when editing the Workflow, you have options to set the concurrency based on your requirements. Set this value higher than the default of 1 if you want to be able to perform multiple runs of the same job concurrently, but with different parameters.

Hope this helps!

-All the Best!

Doug