X-Forwarded-Access-Token occasionally return stale values from a previous session

[tesshy-amane]

Hi all,

I'm running a Streamlit app on Databricks Apps with On-Behalf-Of User Authorization enabled. We've observed that X-Forwarded-Access-Token and X-Forwarded-Email sometimes return values that belong to a previous session(logged out other user, using same browser window) rather than the current one, e.g.:

python

user_access_token = st.context.headers.get("X-Forwarded-Access-Token")
username = st.context.headers.get("X-Forwarded-Email")

username occasionally appears to be a different user's email from an earlier session, and the associated user_access_token then gets used for a Databricks SQL query executed on behalf of the wrong identity.

What we've verified:

• We are not caching the connection or the token anywhere in our code (no st.cache_resource wrapping a connection object, no module-level/global variables holding the token). user_access_token and username are read fresh from st.context.headers at the top of the script on every run.
• Our st.cache_data-decorated functions include username as an explicit cache key argument, so we don't believe this is Streamlit's function-level caching bleeding across users.

Question:

• Is it expected/known that the Databricks Apps reverse proxy (or the underlying connection between the proxy and the app container) can reuse a connection across different end-user sessions in a way that causes X-Forwarded-* headers from one user's request to be forwarded on a subsequent request from a different user?
• Is there a recommended way to positively verify, on every request, that X-Forwarded-Access-Token actually corresponds to X-Forwarded-Email (e.g., calling current_user.me() with the token) before we trust it for a query? We've added this as a client-side safety check in the meantime, but would prefer a platform-level explanation/fix if this is a known proxy behavior.

Any insight into whether this is expected behavior, a known issue, or something we should file a support ticket for would be appreciated. Happy to share more app code / logs if useful.

Thanks!

[iyashk-DB]

This almost always turns out to be st.cache_resource on the connection, not the reverse proxy reusing anything.

You said you checked your st.cache_data functions and they key on username, which is the right instinct, but the leak usually hides in whatever function opens the SQL connection or session object. st.cache_resource caches are globally by default, they're shared across every user and every session in the app process, not per-session like st.session_state. The cache key comes from the function's arguments, so if your connection-builder function looks like this:

@st.cache_resource(ttl=300)
def get_connection():
    return sql.connect(server_hostname=..., http_path=..., access_token=user_access_token)

every user hits the same cached connection object for up to the TTL window, and that connection was opened with whichever token happened to trigger the first cache miss. That's exactly the symptom you're describing: queries occasionally running under a previous user's identity, and it'll look "random" because it only shows up when the cache is warm from someone else's request.

The fix is to make the token part of the cache key so each identity gets its own cached object:

@st.cache_resource(ttl=300)
def get_connection(user_token: str):
    return sql.connect(server_hostname=..., http_path=..., access_token=user_token)

conn = get_connection(user_access_token)

Or simplest and safest, don't cache the connection at all, open it fresh per request and close it after the query, which is what Databricks' own Apps examples do for this exact reason.

To your second question, yes, there's a cheap way to verify the token belongs to who you think before you use it. Use the SDK with that specific token and check the identity it resolves to against X-Forwarded-Email:

from databricks.sdk import WorkspaceClient

w = WorkspaceClient(host=host, token=user_access_token)
me = w.current_user.me()
if me.user_name != username:
    st.error("Session identity mismatch, please refresh.")
    st.stop()

One more thing worth knowing separately from the caching issue: st.context.headers itself is only populated once, at the initial page load / WebSocket connection, and doesn't refresh on reruns. So on a long-lived tab, even without any caching bug, the token you're holding can go stale if the underlying session persists across a reconnect. The identity check above catches that case too, since it fails closed instead of quietly running a query under a token that no longer matches.