cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

InputWidgetNotDefined exception when running a notebook as a job

MarcinO
New Contributor II

I have a notebook that reads a value of a text input in a Scala command:

var startTimeStr = dbutils.widgets.get("Run Date")
What doesn't make any sense that this notebook fails with InputWidgetNotDefined error when being scheduled as a job, but works just fine in interactive mode, on the same cluster. Any advice?
3 REPLIES 3

Kaniz_Fatma
Community Manager
Community Manager

Hi @MarcinOThe issue you're experiencing is that widgets are unsupported in scheduled jobs in Databricks. When you run a notebook interactively, you can set widget values, which can then be accessed via dbutils.widgets.get().

However, when scheduling a job, there's no interactive interface to set these widget values, which is why you're seeing the InputWidgetNotDefined error.

You can pass the parameters directly to the job during scheduling to resolve this issue. You can do this by defining the parameters in the job configuration and then accessing those parameters in your notebook. dbutils.widgets.get().

MarcinO
New Contributor II

I have a number of notebooks that have Widgets and I run them both interactively and in job mode without exceptions. The difference that is these notebooks are in Python, and the one I reported InputWidgetNotDefined exception for is in Scala. So I think there is an inconsistency between Scala and Python runtime here.

The reason I like using Widgets for notebooks that I also run as jobs is that you can set parameters manually during testing or, as you said via job parameters, and you use the same dbutils.widgets.get() API to get the input. It works great in Python commands! But for some reason doesn't work in Scala, that's unfortunate.

berserkersap
Contributor

Have you used dbutils.widget.text() before dbutils.widget.get() ?

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!