I have a Python notebook in Databricks. Within it I have a multiselect widget, which is defined like this:

widget_values = spark.sql(f'''
SELECT my_column
FROM my_table
GROUP BY my_column
ORDER BY my_column
''')

widget_values = widget_values.collect()
widget_values = [i[0] for i in widget_values]

if len(widget_values) >0:
dbutils.widgets.multiselect("My widget", widget_values[0], widget_values)
selection = dbutils.widgets.get("My widget")
else:
print("No data in my_table")

I want the variable 'selection' to be blank if no values are selected in the widget. However, once a value or values has been selected, it gets retained until another selection overwrites it.

For example, say I select the value 'A' in the widget and run the notebook. Selection now = 'A'. I then untick 'A' in the widget and run the notebook again. Selection should = '', but it still = 'A'. This happens even if I delete the value 'A' from the table between the two runs.

I have tried adding this code to the line before selection is chosen, but it doesn't do anything:

selection = ''

I have also tried starting a new session on the compute I'm using to run the notebook. If I do this, no variables appear in the 'variables' pane until I run the notebook again. But if I then run the notebook with no values selected in the widget, suddenly selection = 'A' again.

Does anyone know why Databricks is retaining the selection like this and how I can stop it please?