topic Re: InputWidgetNotDefined exception when running a notebook as a job in Data Engineering

InputWidgetNotDefined exception when running a notebook as a job

MarcinO — Fri, 22 Sep 2023 00:12:23 GMT

I have a notebook that reads a value of a text input in a Scala command:

var startTimeStr = dbutils.widgets.get("Run Date")

What doesn't make any sense that this notebook fails with InputWidgetNotDefined error when being scheduled as a job, but works just fine in interactive mode, on the same cluster. Any advice?

Re: InputWidgetNotDefined exception when running a notebook as a job

MarcinO — Fri, 22 Sep 2023 21:44:30 GMT

I have a number of notebooks that have Widgets and I run them both interactively and in job mode without exceptions. The difference that is these notebooks are in Python, and the one I reported InputWidgetNotDefined exception for is in Scala. So I think there is an inconsistency between Scala and Python runtime here.

The reason I like using Widgets for notebooks that I also run as jobs is that you can set parameters manually during testing or, as you said via job parameters, and you use the same dbutils.widgets.get() API to get the input. It works great in Python commands! But for some reason doesn't work in Scala, that's unfortunate.

Re: InputWidgetNotDefined exception when running a notebook as a job

berserkersap — Sat, 14 Oct 2023 17:52:59 GMT

Have you used dbutils.widget.text() before dbutils.widget.get() ?