cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

How do I pass arguments/variables from widgets to notebooks?

lshar
New Contributor III

Hello,

I am looking for a solution to this problem, which is known since 7 years: https://community.databricks.com/s/question/0D53f00001HKHZfCAP/how-do-i-pass-argumentsvariables-to-n...

What I need is to parametrize my notebooks using widget information. The parameters from the widgets should be passed to a %run of another notebook where all of the use case specific configurations are written.

Hence, I need solution to this problem, where the variable "john" is defined by a widget input.

----Cell1------------------------

john = 10

-----Cell2--------------------

%run path/to/NotebookB $VarA = john

submits "john" to NotebookB not the value of 10

I have found a workaround for this, hovewer the workaround is also having a problem.

Here is an example. As you can see in cmd 6 when I run the notebook with "%run" and a value the notebook is loaded.

In cmd5 I have tried the same, but inside an if-statement, which is failing with the explanation, that the file is missing. However, it is obvious, that the file exists.

example_if_run 

Is anyone having the same issue and knows how to implement this?

1 ACCEPTED SOLUTION

Accepted Solutions

That is a great option to use notebook workflows. I don't think you can pass a variable into the %run commands. It needs to be a literal string.

%run typically cannot access variables/objects from different languages.

One thing to point out is that this method executes in a different session from the parent notebook. %run executes within the same session as the parent notebook.

View solution in original post

7 REPLIES 7

Hubert-Dudek
Esteemed Contributor III

Can't you just use dbutils?

args = {}
args["VarA"] = dbutils.widgets.get("john")
dbutils.notebook.run("NotebookB", timeout=180, args)

That is a great option to use notebook workflows. I don't think you can pass a variable into the %run commands. It needs to be a literal string.

%run typically cannot access variables/objects from different languages.

One thing to point out is that this method executes in a different session from the parent notebook. %run executes within the same session as the parent notebook.

Kaniz
Community Manager
Community Manager

Hi @lshar​ , Did you try the code suggested by @Hubert Dudek​ ?

lshar
New Contributor III

@Hubert Dudek​ @Kaniz Fatma​ 

When I use the dbutils.notebook.run(..) a new cluster is started, hence I can run some other code, but cannot use variable and functions as if I have just run them directly in the same notebook. Hence, my goal is not met.

I want to run a function and use parameters from the notebook that is in the dbutils.notebook.run(), but this is not possible because of running with new cluster.

Ryan_Chynoweth
Honored Contributor III

Ishar, the dbutils.notebook.run() function is used to execute another notebook in a different session on the same cluster. Since the child notebook has a different session the variables, functions, parameters, classes, etc. are not available in the parent notebook.

If you wish to import a function from another notebook I would recommend using the %run functionality as that would execute the child notebook in the same session as the parent notebook.

To achieve your goal please use %run.

Additionally, if you are using python I would also look into our Python Import functionality that is available in Repos.

lshar
New Contributor III

Yes, exactly. This is what I am saying.

I use %run, however I want to pass a parameter, which is selected from the user (or given in a scheduled run) to have different configuration from the notebook, that I am running (example: I want two pass the parameter "female" and load other parameters and functions from the notebook, that I am running with %run, than if I pass "male")

Now the issue is with this parameter as I have already described in the first comment: I pass subtype, but when it is a parameter with value from a widget it is not working.

Got it. So the only way to "pass a parameter" with %run is to define it as a variable in the parent notebook and use that variable in the child notebook. This works because both notebooks are executed in the same session so the variable my_var is available in both notebooks.

Parent Notebook:

my_var = "this is a parameter I want to pass" 
 
%run ./my_child_notebook

Child Notebook:

print(my_var) 
 
>> "this is a parameter I want to pass"

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!