topic Dynamic Parameter in Data Engineering

Dynamic Parameter

SeekingSolution — Tue, 29 Apr 2025 12:40:53 GMT

I have a query I need to run with two parameters: Workflow and workflow steps. The dropdown list supplied by "Steps" should change based on the input of the "Workflow" dropdown.

When I use the following code, it creates the "Steps" dropdown list based on the "Workflow" value that is selected in the parameter at that time. And then the list will not change from there. Is it possible to configure the "Steps" parameter to consistently change the dropdown values as selections for "Workflow" change?

current code:

CREATE WIDGET DROPDOWN Step DEFAULT 'All' CHOICES SELECT stepname from reference.workflowsteps where workflowname in(:Workflow)

Re: Dynamic Parameter

SP_6721 — Wed, 30 Apr 2025 09:35:00 GMT

Hi @SeekingSolution

Databricks widgets don’t support dependent dropdowns automatically. So, when you create a Steps dropdown based on the Workflow selection, it won’t update on its own, you’ll need to manually re-run the cell that builds the Steps widget after selecting a different workflow.

You can set it up by by first creating the Workflow dropdown:

workflows = [row[0] for row in spark.sql("SELECT DISTINCT workflow_name FROM reference.workflowsteps").collect()] dbutils.widgets.dropdown("Workflow", workflows[0], workflows, "Workflow")

And then create the Steps dropdown (re-run this after selecting a Workflow)

selected_workflow = dbutils.widgets.get("Workflow") steps = [row[0] for row in spark.sql(f"SELECT stepname FROM reference.workflowsteps WHERE workflowname = '{selected_workflow}'").collect()] dbutils.widgets.dropdown("Steps", steps[0], steps, "Steps")

Re: Dynamic Parameter

SeekingSolution — Fri, 02 May 2025 18:12:45 GMT

That's a shame it has to be re-instantiated each time! Thank you for letting me know that functionality is not currently supported.