Streamlit Databricks App Compute Scaling

OmarE — Thu, 20 Feb 2025 02:03:41 GMT

I have a streamlit Databricks app and I’m looking to increase the compute resources. According to the documentation and the current settings, the app is limited to 2 vCPUs and 6 GB of memory. Is there a way to adjust these limits or add more resources? I have already added the DATABRICKS_CLUSTER_ID to my environment variables, but it doesn’t seem to affect the compute resources.

Re: Streamlit Databricks App Compute Scaling

mark_ott — Wed, 05 Nov 2025 12:50:40 GMT

You can increase compute resources for your Streamlit Databricks app, but this requires explicitly configuring the compute size in the Databricks app management UI or via deployment configuration—environment variables like DATABRICKS_CLUSTER_ID alone do not change resource limits for your app.

Adjusting Compute Size

Databricks apps have default resource limits of 2 vCPUs and 6 GB of memory, but you can select higher compute sizes for more demanding workloads. To increase these limits, follow these steps:

When creating or editing your app in Databricks, go to the Compute section, select your app, and choose Edit.
In the Configure step, select a larger Compute size from the provided dropdown, such as one offering up to 4 vCPUs and 12 GB of memory.

After saving your changes, your app will gradually switch to the newly selected compute size once the update completes. The active compute size can also be viewed on your app’s Overview tab.

Note on DATABRICKS_CLUSTER_ID

Setting the DATABRICKS_CLUSTER_ID helps your app identify and connect to specific clusters for running jobs or accessing data, but it does not alter the compute resources allocated to Databricks apps themselves. The allocated resources are governed by the compute size you select during app setup or edit—not by environment variables.

Related Guidance

If you need more resources than the available compute sizes, consider using external approaches such as breaking workloads into distributed jobs or moving portions of your workload to Databricks notebooks or jobs where cluster sizes are more flexible.
For persistent performance issues, review your app’s code for memory leaks or inefficient data processing, as resource limits can also be hit due to suboptimal application design.

In summary, you should increase your Databricks app compute resources by editing the app’s configuration and selecting a higher compute size—environment variables alone will not affect these limits.

topic Re: Streamlit Databricks App Compute Scaling in Data Engineering

Streamlit Databricks App Compute Scaling

Re: Streamlit Databricks App Compute Scaling

Adjusting Compute Size

Note on DATABRICKS_CLUSTER_ID

Related Guidance