Pass variable from one notebook to another

ADB0513 — Tue, 06 Aug 2024 15:23:42 GMT

I have a main notebook where I am setting a python variable to the name of the catalog I want to work in. I then call another notebook, using %run, which runs an insert into using a SQL command where I want to specify the catalog using the catalog variable from the main notebook. What is the best way to accomplish this? Is there a way to pass the variable to the notebook that is being called from the %run command? Should I use a widget? Should I use spark.conf?

Thanks in advance for the help.

Re: Pass variable from one notebook to another

Retired_mod — Thu, 08 Aug 2024 08:07:09 GMT

Hi @ADB0513, To pass variables between notebooks in Databricks, you can use three main methods: **Widgets**, where you create and retrieve parameters using `dbutils.widgets` in both notebooks; **spark.conf**, where you set and get configuration parameters; and **Global Variables**, which are less recommended due to scope issues. Widgets and `spark.conf` are generally more robust and maintainable, especially for CI/CD setups.

Would you like more details on any of these methods? Or do you have any other questions about your CI/CD setup with Databricks Asset Bundles?

topic Re: Pass variable from one notebook to another in Data Engineering

Pass variable from one notebook to another

Re: Pass variable from one notebook to another