topic Re: Streamlit app on Databricks doesn't recognise the DATABRICKS_WAREHOUSE_ID in the yaml file in Warehousing & Analytics

Streamlit app on Databricks doesn't recognise the DATABRICKS_WAREHOUSE_ID in the yaml file

AnaMocanu — Mon, 16 Dec 2024 19:59:37 GMT

Hey everyone,

So I managed to create a Streamlit app on Databricks, works fine deployed in the cloud.

However, when I try to run it locally, it complains about "assert os.getenv('DATABRICKS_WAREHOUSE_ID'), "DATABRICKS_WAREHOUSE_ID must be set in app.yaml."" It looks like

valueFrom: "sql-warehouse" is not being picked up properly in the yaml file. In the app config, it created a service principal with the "sql-warehouse" key that should pick up my serverless warehouse that I want to use.

My workaround is to specify the value of the warehouse as an environment variable in the app.py and that seems to work.
like this:

#assert os.getenv('DATABRICKS_WAREHOUSE_ID'), "DATABRICKS_WAREHOUSE_ID must be set in app.yaml."

os.environ['DATABRICKS_WAREHOUSE_ID'] = my_value

Any thoughts on how to enforce the yaml file to pick up the right values?

Many thanks!

Ana

Re: Streamlit app on Databricks doesn't recognise the DATABRICKS_WAREHOUSE_ID in the yaml file

Alberto_Umana — Mon, 16 Dec 2024 20:51:30 GMT

Hi @AnaMocanu,

How are you setting up: DATABRICKS_WAREHOUSE_ID?

When running the app locally, the environment variables set in the app.yaml file might not be picked up automatically. You can manually set the environment variable in your local environment before running the app. For example, in a Unix-based system, you can use

export DATABRICKS_WAREHOUSE_ID=<your_warehouse_id>
streamlit run app.py

Additionally, you might want to implement some prints to validate if information is correct:

import os
print("DATABRICKS_WAREHOUSE_ID:", os.getenv('DATABRICKS_WAREHOUSE_ID'))

Re: Streamlit app on Databricks doesn't recognise the DATABRICKS_WAREHOUSE_ID in the yaml file

AnaMocanu — Mon, 16 Dec 2024 22:53:58 GMT

Hi @Alberto_Umana

Thanks for your reply!
DATABRICKS_WAREHOUSE_ID is set up in yaml file like this:

env:

- name: "DATABRICKS_WAREHOUSE_ID"

valueFrom: "sql-warehouse"

When the app is running on Databricks, I can see a Service principal in app resources that has the value set up.

Your suggestion is how I hack it at the moment, I'm setting up the id manually, but I don't want to do it like that, cause I'm using a serverless warehouse and that value could change?

Re: Streamlit app on Databricks doesn't recognise the DATABRICKS_WAREHOUSE_ID in the yaml file

pdiamond — Mon, 17 Mar 2025 17:22:31 GMT

I'm running into the same problem. Did you ever figure it out?

Re: Streamlit app on Databricks doesn't recognise the DATABRICKS_WAREHOUSE_ID in the yaml file

AnaMocanu — Mon, 17 Mar 2025 19:56:59 GMT

Unfortunately not 😞 the other way to avoid it is to not use serverless as the warehouse name won't change then.

Re: Streamlit app on Databricks doesn't recognise the DATABRICKS_WAREHOUSE_ID in the yaml file

pdiamond — Thu, 20 Mar 2025 14:47:13 GMT

I figured it out this morning. You have to use an underscore in the yaml file instead of a dash. This works for me:

- name: "DATABRICKS_WAREHOUSE_ID"

valueFrom: "sql_warehouse"

Re: Streamlit app on Databricks doesn't recognise the DATABRICKS_WAREHOUSE_ID in the yaml file

AnaMocanu — Thu, 20 Mar 2025 20:08:33 GMT

thanks for that @pdiamond - the underscore didn't work for me, but trying again with the parameter set up it seems that it works now!

I can see the key "sql-warehouse" under App resources. Yours must be set up "sql_warehouse" instead.

Glad we both sorted it.