cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Saving Widgets to Git

shanebo425
New Contributor III

We use Databricks widgets in our python notebooks to pass parameters in jobs but also for when we are running the notebooks manually (outside of a job context) for various reasons. We're a small team, but I've noticed that when I create a notebook and add widgets to it (using the notebook UI but clicking on the Edit menu and selecting Add widget...), those widgets don't seem to save to our git repo for other data engineers or scientists when they pull the notebook from Git. So they have to spend time manually adding the widgets back to the notebook, since the notebook assumes they are there and pulls values for them using `dbutils.widgets.get("<widget-name">)`. 

Is there a setting or something somewhere that I need to turn on so that the widgets persist with the notebook and will automatically be there at the top of the notebook when someone pulls the notebook from git (obviously assuming the open the notebook in the databricks UI and not on their local desktop)? If not, is there another way I should be handling this scenario? Parameterizing our notebooks is necessary as we often reuse them to populate new instances of a catalog/schema or recreate a table, etc. and setting up a job seems like overkill. How do others handle parameterized notebooks that are stored in git and can be open and run by other team members?

2 REPLIES 2

daniel_sahal
Esteemed Contributor

@shanebo425 
You can add your widgets to the code, ex:

dbutils.widgets.text("test", "")
dbutils.widgets.get("test")

 Remember that the cell with widget needs to be run in order for widgets to be actually visible in a notebook.

I did consider that. Is that the only way to ensure these widgets persist for everyone? Will adding these this way affect the notebook if it's being run as a job or will I need to make a copy of the notebook without this extra code for the job version of the notebook (we will often run the same notebooks both manually and as a scheduled job).

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group