Dynamic value input to a job

[Hritik_Moon]

How do I pass dynamic value to a databricks job?

I created a notebook which will extract the names of files in the catalog, I want to pass these names as parameter to another notebook task in a job.

What are the ways I can do this?

[szymon_dybczak]

Hi @Hritik_Moon ,

Define widgets in notebook, so you your notebooks are prepare for passing parameters. Then you can use task values to pass information between tasks. Check below documentation entry for more information and examples:

Use task values to pass information between tasks | Databricks on AWS

[Hritik_Moon]

Hello  I tried this,

Notebook 1 :

dbutils.jobs.taskValues.set(key = "my_key", value = "hi From Notebook1")

Notebook2:

X = dbutils.jobs.taskValues.get(taskKey="01", key="my_key", debugValue = "Fail")

print(X)

 Here I get "Fail" as output, its not fetching my_key

@szymon_dybczak

[szymon_dybczak]

Your task in the workflow is name 01?

Databricks widgets | Databricks on AWS

[szymon_dybczak]

Hi @Hritik_Moon  ,

Folllow my steps. I created 2 notebooks:

- first one called Notebook1 with followign content

- second one called Notebook2 with following content that will read value defined in Notebook1

Here's my definiton of workflow that is using those 2 notebooks:

Pay attention to my taskKey in get method is named the same as the task in a workflow (Notebook1):

[Hritik_Moon]

Hello, I think you have posted wrong picture for notebook1. Could you please verify once.

I made some changes and its working now. 🙂

Thanks a lot.

[szymon_dybczak]

Hi @Hritik_Moon ,

Good catch, I've pasted wrong screenshot. Here's the correct one: