passing job parameters through the terminal to a job

[om_bk_00]

I am having troubles overriding the job parameters that are deployed in my local workspace.

e.g I have a job that fills tables with data,

the parameters given to it are random and I would like to override them when I run through my terminal

databricks bundle run <job_name> <NEW_PARAMETERS>

havent been able to do it yet. But i am able to do it in python using DatabricksRunNowOperator through the json

[VZLA]

Hello, @om_bk_00 thank you for your question.

Could you clarify a few details to help me assist you better? Are you encountering any specific errors when trying to override job parameters via the terminal? Also, are you using the databricks bundle run command exclusively, or have you tried the databricks jobs run-now command? If possible, share the command you're running and the parameters you're trying to pass.

To override parameters via the terminal, the databricks jobs run-now command is typically used:

databricks jobs run-now --job-id <job_id> --notebook-params '{"param1": "value1", "param2": "value2"}'

Replace <job_id> with your job's ID and update the JSON object with the new parameters. Let me know if you’re still having trouble or need help with a specific error!

[om_bk_00]

I was trying to do something like

databricks bundle run xxx-xxx --var="table_name=$source_table"

then I get an error saying that the parameter table_name is not defined, so I guess I misunderstood how overriding parameters work and how "var" is defined.

I will try your method and see if it works 🙂 

[om_bk_00 · Accepted solution]

Ended up using this and it worked 🙂 

databricks bundle run <job_name> --python-params "--param1=$source_table,..."

The only challenge was to figure out how the params needs to be written exactly as they are in the parameterized job; so it took a few trials. Because a list in python looks a bit different than a list in the parameters on databricks (spaces, etc...)

[VZLA]

Excellent! Thanks for sharing your working version and solution!