Hi @odrobek,
Great question -- Databricks Apps is still maturing, and sharing real-world workflows helps everyone. I have been working with Apps across several projects, so here is a rundown of what the production development cycle typically looks like, what works well, and where the friction points are.
DEVELOPMENT WORKFLOW THAT WORKS
The recommended pattern is a three-phase approach: build and test locally in your IDE, run the app locally with browser preview, then deploy to your workspace.
1. Local Development
You can use any IDE that supports Python or Node.js -- VS Code with the Databricks extension is probably the most common choice. The key files in your project are:
- app.yaml -- defines the startup command, environment variables, and required resources (SQL warehouses, jobs, model serving endpoints, secrets, etc.)
- requirements.txt (Python) or package.json (Node.js) -- your dependencies
- Your application source code
A critical concept: apps must use existing Databricks resources and cannot create new ones. You declare resource dependencies in app.yaml, and actual resource IDs are bound at deployment time. This separation enables portability across dev/staging/prod workspaces.
Docs: https://docs.databricks.com/en/dev-tools/databricks-apps/key-concepts.html
2. Supported Frameworks
- Python: Streamlit, Dash, Gradio (plus FastAPI, Flask are pre-installed)
- Node.js: React, Angular, Svelte, Express
The runtime is Ubuntu 22.04 with Python 3.11 and Node.js 22.16.
Docs: https://docs.databricks.com/en/dev-tools/databricks-apps/system-env.html
3. Deployment Options
- Workspace folder (UI): Upload files to workspace, go to Compute > Apps, click Deploy. Simplest but least automatable.
- Workspace folder (CLI): Use "databricks sync --watch" for iterative development, then "databricks apps deploy" to push. Fastest iteration loop.
- Git repository: Deploy directly from GitHub/GitLab/Bitbucket by specifying a branch, tag, or commit SHA. Best for production CI/CD.
- Databricks Asset Bundles: Define your app as a bundle resource with source_code_path, and deploy using "databricks bundle deploy". Ideal for infrastructure-as-code workflows.
Docs: https://docs.databricks.com/en/dev-tools/databricks-apps/deploy.html
COMMON PAIN POINTS (AND WORKAROUNDS)
1. File Size Limit (10 MB per file)
App files cannot exceed 10 MB each -- deployment fails if they do. Workaround: Store large files in Unity Catalog Volumes and load them at runtime.
2. No Persistent State
Apps do not preserve in-memory state after restarts or redeployments. Workaround: Use Databricks SQL tables, Unity Catalog Volumes, or Lakebase for persistence.
3. Logs Disappear When Compute Shuts Down
Databricks does not persist app logs when compute stops. Workaround: Enable App Telemetry (Beta) to write traces, logs, and metrics to Unity Catalog tables.
Docs: https://docs.databricks.com/en/dev-tools/databricks-apps/monitor.html
4. Limited Compute Resources
Two tiers only: Medium (2 vCPUs, 6 GB RAM, 0.5 DBU/hr) and Large (4 vCPUs, 12 GB RAM, 1 DBU/hr). If your app needs heavy computation, offload work to SQL warehouses, Jobs, or Model Serving endpoints.
Docs: https://docs.databricks.com/en/dev-tools/databricks-apps/compute-size.html
5. Authorization Complexity
Two identity models: App authorization (service principal, shared across all users) and User authorization (OAuth SSO, per-user permissions). Understanding when to use which takes time.
Docs: https://docs.databricks.com/en/dev-tools/databricks-apps/key-concepts.html
6. Iteration Speed / Cold Starts
The deploy-test cycle can feel slow. The fastest iteration loop is:
databricks sync --watch . /Workspace/Users/you@company.com/my-app
7. Networking Constraints
- Apps must listen on 0.0.0.0 and use the DATABRICKS_APP_PORT environment variable
- Requests go through a reverse proxy
- Databricks handles TLS -- do not implement your own
- Your app must shut down within 15 seconds of receiving SIGTERM
Docs: https://docs.databricks.com/en/dev-tools/databricks-apps/best-practices.html
8. Workspace Limit of 100 Apps
Hard limit per workspace. For large organizations, this means you need a strategy for workspace allocation.
Docs: https://docs.databricks.com/en/resources/limits.html
WHAT WORKS WELL
- The Databricks SDK integration is seamless. Environment variables for authentication are automatically set, so the SDK "just works" without manual configuration.
- Framework flexibility is good. Supporting both Python and Node.js frameworks covers most use cases.
- Unity Catalog integration for governance and permissions is a significant advantage over standalone deployments.
- Asset Bundles support makes infrastructure-as-code deployment possible, which is critical for production workflows.
- Embedding in external sites via iframe is straightforward.
- Compute size changes do not interrupt service -- the app keeps running on the old tier until the new one is ready.
BEST PRACTICES FOR PRODUCTION
1. Offload heavy processing to SQL warehouses, Jobs, or Model Serving. App compute is optimized for UI rendering, not data crunching.
2. Use in-memory caching (functools.lru_cache, cachetools) for frequent queries to reduce latency and warehouse load.
3. Pin exact dependency versions in requirements.txt to avoid deployment surprises.
4. Use dedicated service principals per app -- do not share credentials across apps.
5. Separate dev/staging/prod using different workspaces and use Asset Bundles to manage promotion.
6. Log to stdout/stderr (not local files) so logs appear in the Databricks UI.
7. Implement global exception handling to prevent crashes from unhandled errors.
8. Use valueFrom for secrets in app.yaml -- never expose raw secrets in environment variables.
Docs: https://docs.databricks.com/en/dev-tools/databricks-apps/best-practices.html
KEY DOCUMENTATION LINKS
- Overview: https://docs.databricks.com/en/dev-tools/databricks-apps/index.html
- Key Concepts: https://docs.databricks.com/en/dev-tools/databricks-apps/key-concepts.html
- System Environment: https://docs.databricks.com/en/dev-tools/databricks-apps/system-env.html
- Configuration: https://docs.databricks.com/en/dev-tools/databricks-apps/configuration.html
- Development: https://docs.databricks.com/en/dev-tools/databricks-apps/app-development.html
- Deployment: https://docs.databricks.com/en/dev-tools/databricks-apps/deploy.html
- Monitoring: https://docs.databricks.com/en/dev-tools/databricks-apps/monitor.html
- Compute Sizing: https://docs.databricks.com/en/dev-tools/databricks-apps/compute-size.html
- Best Practices: https://docs.databricks.com/en/dev-tools/databricks-apps/best-practices.html
- Embedding: https://docs.databricks.com/en/dev-tools/databricks-apps/embed.html
- Asset Bundles: https://docs.databricks.com/en/dev-tools/bundles/resources.html
Would love to hear what specific pain points others have encountered -- and any creative workarounds you have found!
* This reply used an agent system I built to research and draft this response based on the wide set of documentation I have available and previous memory. I personally review the draft for any obvious issues and for monitoring system reliability and update it when I detect any drift, but there is still a small chance that something is inaccurate, especially if you are experimenting with brand new features.
