cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

How to catch exception from dbutils.widgets.get(...)

entimaniac
New Contributor

I'm trying to write python notebook code that can be run from databricks web ui or from airflow. I intend to pass parameters from airflow via the job api using

notebook_params
. From what I understand, these are accessible as widget values.

dbutils.widgets.get
does not appear to accept a default value option so I assume I'll just have to catch an exception in case there isn't a value available. The code thrown on the UI (
InputWidgetNotDefined
) doesn't appear to work when I try catching it:

try:
  test_val = dbutils.widgets.get("test_name")
except InputWidgetNotDefined:
  test_val = "some default value"
print(test_val)

I can do a general, empty

except:
which will work but is generally bad practice to do blanket catches if we can avoid it.

try:
  test_val = dbutils.widgets.get("test_name")
except:
  test_val = "some default value"
print(test_val)

So is there a more precise exception I can catch?

1 REPLY 1

blt
New Contributor II

I handle this exception with something like:

import py4j
 
try:
  value = dbutils.widgets.get("parameter")
except py4j.protocol.Py4JJavaError as e:
  print(e)

If you look more closely at the stack trace you'll see the origin of the message is from some Java code:

/databricks/spark/python/lib/py4j-0.10.9.1-src.zip/py4j/protocol.py in get_return_value(answer, gateway_client, target_id, name)
    325             if answer[1] == REFERENCE_TYPE:
--> 326                 raise Py4JJavaError(
    327                     "An error occurred while calling {0}{1}{2}.\n".
 
Py4JJavaError: An error occurred while calling o318.getArgument.
: com.databricks.dbutils_v1.InputWidgetNotDefined: No input widget named parameter is defined

"com.databricks.dbutils_v1.InputWidgetNotDefined" appears to be a Java class and the actual Python exception to handle is "Py4JJavaError".

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!