topic Re: How do I pass arguments/variables to notebooks? in Data Engineering

How do I pass arguments/variables to notebooks?

__Databricks_Su — Wed, 18 Feb 2015 21:26:52 GMT

Re: How do I pass arguments/variables to notebooks?

__Databricks_Su — Wed, 18 Feb 2015 21:26:52 GMT

Executing NotebookB from NotebookA with arguments, you would use the following syntax within NotebookA to define the arguments:

%run path/to/NotebookB $VarA="ValueA" $VarB="ValueB"

Within NotebookB, you'd use the following to receive the argument value:

Scala and Python:

print getArgument("VariableName", "DefaultValue")

SQL (No default value is supported):

select * from $VariableName

Re: How do I pass arguments/variables to notebooks?

haidar — Fri, 10 Apr 2015 19:06:39 GMT

should'nt the SQL be

select * from $VarA

Re: How do I pass arguments/variables to notebooks?

kruhly — Thu, 04 Jun 2015 23:11:24 GMT

Is there a way to pass a variable's value from another cell to NotebookB? For example,

----Cell1------------------------

john = 10

-----Cell2--------------------

%run path/to/NotebookB $VarA = john

submits "john" to NotebookB not the value of 10

Re: How do I pass arguments/variables to notebooks?

arsalan1 — Fri, 05 Jun 2015 00:16:59 GMT

@kruhly Unfortunately right now you can't pass specific variable names, only constants. It's on our roadmap to improve/overhaul parameter passing, which would include this capability.

Re: How do I pass arguments/variables to notebooks?

oren1_558848696 — Fri, 23 Oct 2015 15:47:32 GMT

Is there any plan to support default argument values for SQL?

Re: How do I pass arguments/variables to notebooks?

SimeonSimeonov_ — Fri, 08 Jan 2016 23:49:04 GMT

@Oren Benjamin you can use a UDF in SQL and then, in the UDF's code, you can use the Scala/Python form.

Re: How do I pass arguments/variables to notebooks?

HananShteingar1 — Fri, 20 May 2016 12:42:54 GMT

@Databricks_Support, When I try the above, first %run tells me it cannot find the notebook (albeit they are in the same folder. I do not know how to set path\to\notebook right). Second, the getArgument gives me an error

py4j.Py4JException: Method createTextWidget([class java.lang.String, class java.lang.Integer, null, null]) does not exist

. I am using Python notebooks.

Re: How do I pass arguments/variables to notebooks?

nmud19 — Thu, 04 Aug 2016 11:51:57 GMT

Is there any similir functionality for an R notebook?

Re: How do I pass arguments/variables to notebooks?

naman1994 — Tue, 10 Apr 2018 05:24:05 GMT

If you are running a notebook from another notebook, then use dbutils.notebook.run(path = " ", args={}, timeout='120'), you can pass variables in args = {}. And you will use dbutils.widget.get() in the notebook to receive the variable.

And if you are not running a notebook from another notebook, and just want to a variable before running particular cell in the notebook, use dbutils.widget.text("a","b","c") and dbutils.widget.get().

dbutils.widget.text("a","b","c") :Set the value; a,b,c are mandatory, a : name of the variable, b : default value, c : name of the textbox

dbutils.widget.get(): Assign the value to variable, I am not sure about the arguments here, look it up.

Re: How do I pass arguments/variables to notebooks?

lshar — Mon, 28 Mar 2022 16:43:06 GMT

Hello, is there already a solution to this problem?

I have found a workaround for my use case. Hovewer, the workaround seems to have a databricks failure since without using the if statement the %run code is working and with if - an error is displayed as can be seen in the screenshot.

I am very interested in a solution!

Re: How do I pass arguments/variables to notebooks?

osr2020 — Fri, 03 Feb 2023 00:47:27 GMT

where there any updates in this regard?

it's been 1.5 years, so i'm hoping this capability was added by now.

Re: How do I pass arguments/variables to notebooks?

Meghala — Sat, 04 Feb 2023 14:49:08 GMT

This info also needed to me as well

Re: How do I pass arguments/variables to notebooks?

Databricks_-Dat — Wed, 08 Feb 2023 11:57:31 GMT

Looks like this feature is still not available. is there any plan to pass parameter values from another cell when running the notebook via %run?

Re: How do I pass arguments/variables to notebooks?

Manoj12421 — Thu, 09 Feb 2023 19:12:42 GMT

You can use % run and then provide exact path of another notebook which one you want to call and for passing parameter in Pyspark just use Variable1= 'Valuea'

Variable2= 'Valueb'

For scala use :

$ Vara='Valuea'

$ Varb=' Valueb'

Re: How do I pass arguments/variables to notebooks?

Dior — Wed, 26 Apr 2023 16:00:04 GMT

Using dbutils.notebook api helped with this. The command follows this syntax:

dbutils.notebook.run("notebook", "timeout_secounds", {"widget_value": " "})

To get a variable in there just do this:

x = "value"

dbutils.notebook.run("run_weekly_notebook", 60, {"stores_available": f"{x}"})

Re: How do I pass arguments/variables to notebooks?

luis_herrera — Thu, 27 Apr 2023 20:08:03 GMT

You can use a JSON file to temporarily store the arguments that you want to use in your notebook when passing arguments/variables to it. Using json.You can define the argument list and convert it to a JSON file ().Once the argument file is created, you can open it in a notebook and use the arguments by reading the contents and converting the JSON back into a dictionary with json.loads().

https://kb.databricks.com/jobs/pass-arguments-to-a-notebook-as-a-list

Re: How do I pass arguments/variables to notebooks?

luis_herrera — Fri, 28 Apr 2023 15:50:04 GMT

To pass arguments/variables to a notebook, you can use a JSON file to temporarily store the arguments and then pass it as one argument to the notebook. After passing the JSON file to the notebook, you can parse it with json.loads(). The argument list should be defined and converted to a JSON file using json.dumps(). Once the argument file is created, you can open it inside a notebook and use the arguments by reading the contents and converting the JSON back into a dictionary using json.loads().

PS: Check #DAIS2023 talks