How do I pass arguments/variables from widgets to notebooks?

[lshar]

Hello,

I am looking for a solution to this problem, which is known since 7 years: https://community.databricks.com/s/question/0D53f00001HKHZfCAP/how-do-i-pass-argumentsvariables-to-n...

What I need is to parametrize my notebooks using widget information. The parameters from the widgets should be passed to a %run of another notebook where all of the use case specific configurations are written.

Hence, I need solution to this problem, where the variable "john" is defined by a widget input.

----Cell1------------------------

john = 10

-----Cell2--------------------

%run path/to/NotebookB $VarA = john

submits "john" to NotebookB not the value of 10

I have found a workaround for this, hovewer the workaround is also having a problem.

Here is an example. As you can see in cmd 6 when I run the notebook with "%run" and a value the notebook is loaded.

In cmd5 I have tried the same, but inside an if-statement, which is failing with the explanation, that the file is missing. However, it is obvious, that the file exists.

 

Is anyone having the same issue and knows how to implement this?

[Hubert-Dudek]

Can't you just use dbutils?

args = {}
args["VarA"] = dbutils.widgets.get("john")
dbutils.notebook.run("NotebookB", timeout=180, args)

[Ryan_Chynoweth · Accepted solution]

That is a great option to use notebook workflows. I don't think you can pass a variable into the %run commands. It needs to be a literal string.

%run typically cannot access variables/objects from different languages.

One thing to point out is that this method executes in a different session from the parent notebook. %run executes within the same session as the parent notebook.

[Kaniz_Fatma]

Hi @lshar​ , Did you try the code suggested by @Hubert Dudek​ ?

[lshar]

@Hubert Dudek​ @Kaniz Fatma​ 

When I use the dbutils.notebook.run(..) a new cluster is started, hence I can run some other code, but cannot use variable and functions as if I have just run them directly in the same notebook. Hence, my goal is not met.

I want to run a function and use parameters from the notebook that is in the dbutils.notebook.run(), but this is not possible because of running with new cluster.

[Ryan_Chynoweth]

Ishar, the dbutils.notebook.run() function is used to execute another notebook in a different session on the same cluster. Since the child notebook has a different session the variables, functions, parameters, classes, etc. are not available in the parent notebook.

If you wish to import a function from another notebook I would recommend using the %run functionality as that would execute the child notebook in the same session as the parent notebook.

To achieve your goal please use %run.

Additionally, if you are using python I would also look into our Python Import functionality that is available in Repos.

[lshar]

Yes, exactly. This is what I am saying.

I use %run, however I want to pass a parameter, which is selected from the user (or given in a scheduled run) to have different configuration from the notebook, that I am running (example: I want two pass the parameter "female" and load other parameters and functions from the notebook, that I am running with %run, than if I pass "male")

Now the issue is with this parameter as I have already described in the first comment: I pass subtype, but when it is a parameter with value from a widget it is not working.

[Ryan_Chynoweth]

Got it. So the only way to "pass a parameter" with %run is to define it as a variable in the parent notebook and use that variable in the child notebook. This works because both notebooks are executed in the same session so the variable my_var is available in both notebooks.

Parent Notebook:

my_var = "this is a parameter I want to pass" 
 
%run ./my_child_notebook

Child Notebook:

print(my_var) 
 
>> "this is a parameter I want to pass"