cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Question regarding Databricks Apps deployment

Khasim_1
Contributor

Hello Databricks Community,

I am currently working on setting up a Databricks App and ran into a few questions.

Here are the details of what I am working with:

  • Target Framework: [e.g., Streamlit / Dash]
  • Goal: [Briefly describe what you are trying to achieve]
  • Current Challenge: [Briefly describe the blocker, e.g., authentication, Unity Catalog connection, etc.]

Has anyone tackled a similar setup?

Any pointers, code snippets, or best practices would be greatly appreciated!

Data Architect | 13 Years Domain Expertise | Databricks SA Champion Cohort
3 REPLIES 3

data_pulse
Contributor

@Khasim_1 

When I was developing a Flask-based Databricks App, which integrated Databricks Model Serving, a custom tool-calling agent, and Unity Catalog system tables, I found a few key areas worth considering during setup.

  • Application startup: verify app.yaml dependencies, startup command and port configuration.
  • Authentication and permissions: Check which identity the app uses and whether it has access to the required Databricks resources.
  • Unity Catalog access: Confirm the necessary catalog/schema/table privileges.
  • Resource dependencies: Ensure any SQL warehouses or serving endpoints are correctly configured and accessible.
  • Troubleshooting: Deployment/runtime logs to distinguish startup errors from resource-access failures.

In my implementation, the App successfully queried job, billing and query metrics and integrated them with an AI agent, so this architecture is workable.

Could you share your framework (Streamlit, Dash or Flask), what you're trying to build, and the specific error or blocker? That would help to contribute further.

 

john21cornell
New Contributor

When setting up Databricks Apps, common challenges center on configuration, Unity Catalog (UC) connectivity, and authentication. Ensure your app.yaml correctly specifies the startup command and binds to the container port, and leverage the Databricks SDK (databricks-sdk) with a serverless SQL Warehouse so your app can seamlessly access UC datasets using the platform's automatically injected OAuth environment variables. If you share your specific framework, objective, or error details, the community can provide tailored code snippets and debugging guidance.

Isaac_18
Contributor

Hi @Khasim_1,

It looks like the template placeholders made it into the post, so I'm not sure exactly where you're stuck. In case it helps, here's a minimal Streamlit app that reads a Unity Catalog table through a SQL warehouse. That covers the three things people usually get stuck on: app.yaml, authentication and UC permissions.

1. Files

my-app/
  app.py
  app.yaml
  requirements.txt

app.yaml (the warehouse ID comes from an app resource with the key sql-warehouse):

command: ['streamlit', 'run', 'app.py']
env:
  - name: 'DATABRICKS_WAREHOUSE_ID'
    valueFrom: 'sql-warehouse'
  - name: 'STREAMLIT_GATHER_USAGE_STATS'
    value: 'false'

requirements.txt:

databricks-sdk
databricks-sql-connector
pandas

app.py:

import os
import streamlit as st
from databricks import sql
from databricks.sdk.core import Config

cfg = Config()  # picks up the app's own service principal credentials

@st.cache_resource
def get_connection():
    return sql.connect(
        server_hostname=cfg.host,
        http_path=f"/sql/1.0/warehouses/{os.getenv('DATABRICKS_WAREHOUSE_ID')}",
        credentials_provider=lambda: cfg.authenticate,
    )

def run_query(query: str):
    with get_connection().cursor() as cursor:
        cursor.execute(query)
        return cursor.fetchall_arrow().to_pandas()

st.title("Quick table viewer")
df = run_query("SELECT * FROM samples.nyctaxi.trips LIMIT 100")
st.dataframe(df)

2. Add the warehouse as a resource

When you create or edit the app, add a SQL warehouse resource, give it the key sql-warehouse and the Can use permission. That's what fills in DATABRICKS_WAREHOUSE_ID, and it gives the app's service principal access to the warehouse.

3. Grant Unity Catalog access to the app's service principal

By default the app queries as its own service principal, not as you. That's the most common reason an app works in testing and then hits "permission denied". You can find the principal on the app's Authorization tab. Use its application ID in the grants:

GRANT USE CATALOG ON CATALOG main TO `<app-sp-application-id>`;
GRANT USE SCHEMA  ON SCHEMA  main.sales TO `<app-sp-application-id>`;
GRANT SELECT      ON TABLE   main.sales.orders TO `<app-sp-application-id>`;

If you need each user to see only what they're allowed to see, look at user authorization (on-behalf-of-user) in the Apps docs instead.

4. Deploy

databricks apps create my-app
databricks sync ./my-app /Workspace/Users/<you>/my-app
databricks apps deploy my-app --source-code-path /Workspace/Users/<you>/my-app

If it doesn't start, check the Logs tab on the app page first. A missing package in requirements.txt or a wrong command in app.yaml shows up there right away.

If you share your framework, what you're building and the actual error, I'm happy to look at the specific blocker.

Docs: