Retrieve job-level parameters in Python

[John_Rotenstein]

Parameters can be passed to Tasks and the values can be retrieved with:

dbutils.widgets.get("parameter_name")

More recently, we have been given the ability to add parameters to Jobs.

However, the parameters cannot be retrieved like Task parameters.

Question: How can we retrieve job-level parameters in notebook code?

[cbern]

@Retired_mod This method works for Task parameters. Is there a way to access Job parameters that apply to the entire workflow, set under a heading like this in the UI:

I am able to read Job parameters in a different way from Task parameters using  dynamic value references:

{{tasks.[task_name].values.[value_name]}}

vs.

{{job.parameters.[name]}} 

This works for reading Parameters in the Workflow itself, such as:

Is there an analogous way to read a Job parameter within a Notebook? The Note on this page seems to indicate that these dynamic value references are available in Notebooks, but how do you reference them in Python?

[cbern]

A coworker has answered this question for me, posting it for anyone else looking for an answer:

run_parameters = dbutils.notebook.entry_point.getCurrentBindings() 
context = json.loads(dbutils.notebook.entry_point.getDbutils().notebook().getContext().toJson()) 
job_param = run_parameters[key]

[cbern]

an update to my answer: Databricks has advised us that the `dbutils.notebook.entry_point` method is not supported (could be deprecated), and the recommended way to read in a job parameter is through widgets, i.e. `dbutils.widgets.get("param_key")` (similar to Task parameters -- if you have a Task param and Job param with the same name, the Job param value takes precedence). 

If you want to read a dynamic value reference like the run_id inside a Notebook, you can set the reference (e.g. `{{job.run_id}}`) a Task or Job param, and read it like a widget.

[xiangzhu]

I think `dbutils.widgets.get` is for notebooks only, not for python job as asked by this thread

[xiangzhu]

ah sorry, the thread asked for notebooks too.

nevertheless, I'm search for getting job params in pure python jobs

[julio_resende]

You need to push down your parameters to a task level. Eg:

1. Create a job level parameter called "my_param"
2. Make a reference to his job parameter in the task level parameters box. Eg:
   
   ["--my_param","{{job.parameters.my_param}}"]
3. Read the task level parameter using argparser in your .py file

[xiangzhu]

Thanks !

[lprevost]

The only thing that has worked for me consistently in python is 

params = dbutils.widgets.getAll() where an empty dictionary is returned if I'm in interactive mode and the job/task params are returned if they are present.