Databricks workflow job

[ctiwari7]

Hi team,

I am trying to execute a workflow job which takes in a parameter as unique identifier. I am using this job parameter to push down to tasks. I was hoping if there is any way for me to use python uuid4() function to generate unique id every time the workflow job runs. 

Is there any way to generate uuid on the run time as job parameter.?

[Walter_C]

The intention is that the same job generates this id and once it is generated that this id is passed as a parameter for other tasks?

[Stefan-Koch]

hi ctiwari7

A possible way to do that, you create a python file which generates the uuid and then pass it to jobs.taskValues. This is described here: https://docs.databricks.com/en/jobs/task-values.html 

As test, I created a python file, with the following content:

import uuid

uuid = str(uuid.uuid4())

print(uuid)

dbutils.jobs.taskValues.set(key = "uuid", value = uuid)

The file will be added as task in a databricks job.

For the demo, I have a simple notebook which takes a widget as parameter:

In the job, I add a second task, where I refer to the notebook:

In the parameter field: put the following information:
name: uuid
value:  {{tasks.create_uuid.values.uuid}}