How to stop Databricks retaining widget selection between runs?

[SRJDB]

I have a Python notebook in Databricks. Within it I have a multiselect widget, which is defined like this:

widget_values = spark.sql(f'''
SELECT my_column
FROM my_table
GROUP BY my_column
ORDER BY my_column
''')

widget_values = widget_values.collect()
widget_values = [i[0] for i in widget_values]

if len(widget_values) >0:
dbutils.widgets.multiselect("My widget", widget_values[0], widget_values)
selection = dbutils.widgets.get("My widget")
else:
print("No data in my_table")

I want the variable 'selection' to be blank if no values are selected in the widget. However, once a value or values has been selected, it gets retained until another selection overwrites it.

For example, say I select the value 'A' in the widget and run the notebook. Selection now = 'A'. I then untick 'A' in the widget and run the notebook again. Selection should = '', but it still = 'A'. This happens even if I delete the value 'A' from the table between the two runs.

I have tried adding this code to the line before selection is chosen, but it doesn't do anything:

selection = ''

I have also tried starting a new session on the compute I'm using to run the notebook. If I do this, no variables appear in the 'variables' pane until I run the notebook again. But if I then run the notebook with no values selected in the widget, suddenly selection = 'A' again.

Does anyone know why Databricks is retaining the selection like this and how I can stop it please?

[Louis_Frolio · Accepted solution]

Hello @SRJDB ,

What you’re running into isn’t your Python variable misbehaving—it’s the widget hanging onto its own internal state. A Databricks widget will happily keep whatever value you gave it, per user and per notebook, until you explicitly clear or overwrite it. So when you do selection = '' in Python, that doesn’t touch the widget’s bound value returned by dbutils.widgets.get(). To make things more fun, widgets also have a long-standing quirk where state doesn’t always clear after a Run All, which explains why the panel and the printed value sometimes disagree.

Let’s dig into why this happens.

• dbutils.widgets.get() always returns the widget’s stored value, not whatever Python variable you decide should be “the truth.”

• Multiselect widgets send back a comma-delimited string. If nothing is selected it should be empty—unless the widget resurrects a previous state or you recreate it with a baked-in default.

• And that default matters. With defaultValue=widget_values[0], you’ve guaranteed that every redefinition forces a selection. There’s no way to end up blank unless you explicitly allow one.

• Bonus: after a Run All, widgets sometimes refuse to clear state, even if you remove/reset them in code. Running the cells individually usually sidesteps that. If you ever need fully consistent state behavior, ipywidgets are the more predictable path.

Now here are the patterns that actually work.

Option 1: Remove and recreate the widget before reading it.

And yes—removal must happen in a different cell.

# Cell 1: Reset widget
try:
    dbutils.widgets.remove("My widget")
except Exception:
    pass

# Cell 2: Recreate and read
widget_values = spark.sql("""
  SELECT my_column
  FROM my_table
  GROUP BY my_column
  ORDER BY my_column
""").collect()
widget_values = [r[0] for r in widget_values]

choices = [""] + widget_values  # Explicit blank
dbutils.widgets.multiselect("My widget", "", choices)

raw = dbutils.widgets.get("My widget")
selection_list = [s for s in raw.split(",") if s.strip()]
selection = "" if not selection_list else ",".join(selection_list)

Option 2: Add a labeled sentinel (“”) and filter it out cleanly.

choices = ["<none>"] + widget_values
dbutils.widgets.multiselect("My widget", "<none>", choices)

raw = dbutils.widgets.get("My widget")
selection_list = [s for s in raw.split(",") if s and s != "<none>"]
selection = "" if not selection_list else ",".join(selection_list)

Option 3: Stop resetting the default to the first value.

This line guarantees a selection every time:

dbutils.widgets.multiselect("My widget", widget_values[0], widget_values)

If your goal is “blank unless chosen,” this works against you.

A few extra notes:

• If Run All bites you, run the reset/create cells manually or move to ipywidgets where state behaves like you expect.

• Widgets speak strings only; multiselects always require you to parse the comma-delimited output.

• Notebook jobs ignore whatever’s in the UI—they start with defaults unless you pass overrides.

Hope this helps, Louis.