a month ago - last edited a month ago
I am having troubles overriding the job parameters that are deployed in my local workspace.
e.g I have a job that fills tables with data,
the parameters given to it are random and I would like to override them when I run through my terminal
databricks bundle run <job_name> <NEW_PARAMETERS>
havent been able to do it yet. But i am able to do it in python using DatabricksRunNowOperator through the json
4 weeks ago - last edited 4 weeks ago
Ended up using this and it worked ๐
databricks bundle run <job_name> --python-params "--param1=$source_table,..."
The only challenge was to figure out how the params needs to be written exactly as they are in the parameterized job; so it took a few trials. Because a list in python looks a bit different than a list in the parameters on databricks (spaces, etc...)
a month ago
Hello, @om_bk_00 thank you for your question.
Could you clarify a few details to help me assist you better? Are you encountering any specific errors when trying to override job parameters via the terminal? Also, are you using the databricks bundle run command exclusively, or have you tried the databricks jobs run-now command? If possible, share the command you're running and the parameters you're trying to pass.
To override parameters via the terminal, the databricks jobs run-now command is typically used:
databricks jobs run-now --job-id <job_id> --notebook-params '{"param1": "value1", "param2": "value2"}'
Replace <job_id> with your job's ID and update the JSON object with the new parameters. Let me know if youโre still having trouble or need help with a specific error!
a month ago - last edited a month ago
I was trying to do something like
4 weeks ago - last edited 4 weeks ago
Ended up using this and it worked ๐
databricks bundle run <job_name> --python-params "--param1=$source_table,..."
The only challenge was to figure out how the params needs to be written exactly as they are in the parameterized job; so it took a few trials. Because a list in python looks a bit different than a list in the parameters on databricks (spaces, etc...)
4 weeks ago
Excellent! Thanks for sharing your working version and solution!
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโt want to miss the chance to attend and share knowledge.
If there isnโt a group near you, start one and help create a community that brings people together.
Request a New Group