How to pass job task parameters to another task in scala
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-30-2022 10:56 AM
Hi Team,
I have a requirement in workflow job. Job has two tasks, one is python-task and another one is scala-task (both are running their own cluster).
I have defined dbutils.job.taskValue in python which is not able to read value in scala because of job.taskValue is not supported in scala.
So, the question is,
how to pass custom parameters to task 2 ( scala )?.
Is dbutil.widget can work here by setText in python-task and read widget.get in scala-task ?.
Do we have any approaches?.
Note : We shouldn't use notebook.run.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-30-2022 11:10 AM
Hi @Sreekanth Nallapa
Assuming that you want to pass python notebook params to scala nb. Else dbutils.jobs.taskValues is also available in scala.
If you want to pass parameters from py nb to scala nb or vice-versa,
try using dbutils.notebook.exit() and for the returnValue part, return the parameter you want.
You can then pull the exited value into your other notebook.
Cheers..
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-06-2022 01:56 AM
