Databricks Community

MrChromatic · a month ago

Hi everyone,

I’m deploying a frontend (Streamlit) and backend (FastAPI) as two separate Databricks Apps within the same workspace, both with user authentication enabled.

The frontend makes a server-side HTTP request to the backend app URL when a user submits a request. However, instead of receiving a JSON response from the backend, the request is intercepted by the Databricks authentication layer and returns the Databricks Sign In HTML page.

This HTML response then causes JSON parsing to fail on the frontend side. I’d like to understand whether deploying frontend and backend as two separate Databricks Apps is the recommended pattern for this type of setup, and how inter-app communication is expected to work when the authentication layer intercepts backend calls made programmatically from another app.

SteveOstrowski · 2 weeks ago

Hi @MrChromatic,

The behavior you are seeing is expected. When your Streamlit frontend app makes an HTTP request to your FastAPI backend app URL, that request goes through the Databricks authentication proxy just like any browser request would. Since the server-side HTTP call from your frontend does not carry a valid Databricks session or token, the auth layer intercepts it and returns the sign-in HTML page instead of your backend's JSON response.

Here is what is happening and how to resolve it.

WHY THIS HAPPENS

Each Databricks App with user authentication enabled sits behind an authentication reverse proxy. When a request arrives at the app URL, the proxy checks for a valid Databricks OAuth session. Your frontend app's server-side HTTP call (e.g., via the requests library) does not include the user's Databricks session cookies or access token, so the proxy redirects to the sign-in page. Your frontend then tries to parse that HTML as JSON and fails.

HOW TO FIX IT

You need to forward the user's access token from your frontend app to your backend app. Databricks Apps automatically injects the authenticated user's token via the x-forwarded-access-token HTTP header into requests that reach your app. Your frontend can read this token and include it when calling the backend.

STEP 1: Read the user token in your Streamlit frontend

In your Streamlit app, retrieve the token from the request headers:

import streamlit as st

user_access_token = st.context.headers.get("x-forwarded-access-token")

STEP 2: Pass the token when calling your backend

Include the token as a Bearer token in the Authorization header when making requests to your FastAPI backend:

import requests

backend_url = "https://<your-backend-app-url>/your-endpoint"
headers = {
  "Authorization": f"Bearer {user_access_token}",
  "Content-Type": "application/json"
}
response = requests.post(backend_url, json=payload, headers=headers)
result = response.json()

STEP 3: Receive the token in your FastAPI backend

In your FastAPI backend, the token arrives in the standard Authorization header. You can then use it to make further Databricks API calls on behalf of the user, or simply validate that the caller is authenticated:

from fastapi import FastAPI, Request

app = FastAPI()

@app.post("/your-endpoint")
async def handle_request(request: Request):
  auth_header = request.headers.get("Authorization")
  # auth_header will be "Bearer <token>"
  # Use the token for downstream Databricks API calls if needed
  return {"status": "success", "data": "your response"}

ALTERNATIVE: USE THE APP'S OWN SERVICE PRINCIPAL

If you do not need user-level identity for your backend calls, you can configure your backend app to skip user authentication and instead rely solely on its own service principal identity. Every Databricks App gets a dedicated service principal with credentials available via the DATABRICKS_CLIENT_ID and DATABRICKS_CLIENT_SECRET environment variables.

In this pattern, your frontend app would authenticate the user, and then call the backend using a service-to-service credential flow. Your frontend can generate an M2M OAuth token using its own service principal credentials:

from databricks.sdk.core import Config

cfg = Config()
# cfg.authenticate() returns headers with a valid Bearer token
auth_headers = cfg.authenticate()
response = requests.post(backend_url, json=payload, headers=auth_headers)

For this to work, you would need to grant your frontend app's service principal CAN_MANAGE or CAN_USE permission on the backend app.

RECOMMENDED ARCHITECTURE

Consider whether you truly need two separate apps. If the backend is only serving your frontend, you could consolidate both into a single Databricks App by running FastAPI as your backend and mounting Streamlit or serving the frontend from the same process. This avoids the cross-app authentication challenge entirely.

If you do need separate apps (for example, because other clients also call the backend), the token-forwarding approach described above is the way to go.

USEFUL REFERENCES

Databricks Apps authorization documentation:
https://docs.databricks.com/aws/en/dev-tools/databricks-apps/auth.html

Databricks Apps HTTP headers documentation:
https://docs.databricks.com/aws/en/dev-tools/databricks-apps/http-headers.html

Databricks Apps authorization demo on GitHub:
https://github.com/databricks-solutions/databricks-apps-examples/tree/main/auth-demo

Databricks Apps best practices:
https://docs.databricks.com/en/dev-tools/databricks-apps/best-practices.html

* This reply used an agent system I built to research and draft this response based on the wide set of documentation I have available and previous memory. I personally review the draft for any obvious issues and for monitoring system reliability and update it when I detect any drift, but there is still a small chance that something is inaccurate, especially if you are experimenting with brand new features.

If this answer resolves your question, could you mark it as "Accept as Solution"? That helps other users quickly find the correct fix.

View solution in original post

Lu_Wang_ENB_DBX · 3 weeks ago

Summary Deploying a frontend and backend as separate Databricks Apps is a supported architectural pattern. To resolve the authentication interception, your frontend app must authenticate its server-to-server requests using its dedicated service principal (M2M OAuth) rather than relying on user authentication.

How Inter-App Communication Works Your frontend is receiving the Databricks Sign-In HTML page because the backend app's user authentication layer is intercepting the unauthenticated server-side HTTP request.

To fix this, you need to leverage the dual identity models provided by Databricks Apps:

Dual Identity Model: Databricks Apps support both user authentication (where users log in with their credentials) and service principal authentication (where the app uses its dedicated service principal). You can mix and match these authorization models within the same app.
Service Principal Authentication: Because the frontend is making a server-side request to the backend, it cannot rely on the end-user's session. The frontend app must authenticate using its own dedicated service principal.
Implementation: You need to grant the frontend app's service principal access to the backend app. The frontend app must then fetch its service principal OAuth token and pass it in the headers of the HTTP request. This will bypass the user login screen and allow the FastAPI backend to successfully return the JSON response.

SteveOstrowski · 2 weeks ago