01-24-2022 11:17 AM
I'm running a databricks job involving multiple tasks and would like to run the job with different set of task parameters. I can achieve that by edit each task and and change the parameter values. However, it gets very manual when I have a lot of tasks. I'm think of using "run now with different parameters" option and pass in a different job JSON file. However, it looks like no matter what pass into the JSON file at the job level, my tasks still run with the old parameter values.
Does "run now with different parameters" option support changing parameter values for several tasks at once?
01-25-2022 06:51 AM
Hello, @Shirly Wang! My name is Piper, and I'm a moderator for the Databricks community. Welcome to the community and thank you for your question! It's nice to meet you. Let's give the members of the community a chance to respond before we return to your question.
Thanks for your patience.
01-28-2022 08:56 AM
hi @Shirly Wang , To Run a job with different parameters,
You can use Run Now with Different Parameters to re-run a job with different parameters or different values for existing parameters.
2. Click Run.
08-25-2022 07:36 AM
This is an issue for me too. I am using a multi-stage job calling the same notebook with different values for the parameters. This feels like a bug to me.
11-04-2022 10:28 AM
Hi @Alastair Reilly , I was checking to see if my suggestions helped you.
Or else, If you have any solution, please share it with the community as it can be helpful to others.
Also, Please don't forget to click on the "Select As Best" button whenever the information provided helps resolve your question.
12-14-2022 04:32 AM
Hello,
I am also facing with the same issue. The problem is described below:
Apart from Jobs UI, I also suffer from the same problem when I want to trigger the job using API (reference).
In short, how can I pass different parameters to tasks and get those parameters merged instead of being overridden in a multi-task environment?
Excited to expand your horizons with us? Click here to Register and begin your journey to success!
Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!