topic Re: Pass through if a job was run as scheduled or if manual in Data Engineering

Pass through if a job was run as scheduled or if manual

cmilligan — Tue, 29 Nov 2022 17:25:06 GMT

I have a notebook that sets up parameters for the run based on some job parameters set by the user as well as the current date of the run. I want to supersede some of this logic and just use the manual values if kicked off manually. Is there a way to pass through if a job was kicked off manually or scheduled?

Re: Pass through if a job was run as scheduled or if manual

UmaMahesh1 — Tue, 29 Nov 2022 17:32:35 GMT

Hi @Coleman Milligan ,

One way to try would be to use widgets in the notebooks you are using setting the widget.get default values to be your job parameters. If you are running the job manually, whatever parameters you provide to the job manually would be taken and used.

Hope my understanding of your requirement was clear.

Cheers..

Re: Pass through if a job was run as scheduled or if manual

SS2 — Tue, 29 Nov 2022 20:33:07 GMT

You can create widgets by using this- dbutils.widgets.text("widgetName", "")

To get the value for that widget:- dbutils.widgets.get("widgetName")

So by using this you can manually create widgets (variable) and can run the process by giving desired value.

If you want to take the value of widgets from the job parameter then you can also do.

Re: Pass through if a job was run as scheduled or if manual

Sreekanth1 — Wed, 30 Nov 2022 19:07:45 GMT

Can we set widget.text in one task notebook and another task notebook can get that value in workflow job ?