Long-lived authentication for Databricks Apps / FastAPI when using Service Principal (IoT use case)

[Punit_Prajapati]

Hi Community,

I’m working with Databricks Apps (FastAPI) and invoking the API from external IoT devices.

Currently, the recommended approach is to authenticate using a Bearer token generated via a Databricks Apps Service Principal (Client ID + Client Secret), as documented here:
https://apps-cookbook.dev/docs/fastapi/getting_started/connections/connect_from_external

This works, but the generated access token expires in 3600 seconds (1 hour).
For my use case (IoT devices deployed in the field), this is not practical, as I need a long-lived or non-expiring token, or at least a mechanism that does not require frequent re-authentication.

What I’ve tried

1. Using Service Principal OAuth flow to generate the Bearer token

   • Token expires after 1 hour (expected behavior)

2. Exploring on-behalf-of (OBO) token creation as discussed here:
   https://community.databricks.com/t5/data-engineering/on-behalf-of-token-creation-for-spn/td-p/32451

   • I have already granted CAN_MANAGE permissions on the Service Principal to my user

   • However, this approach does not work for my scenario because In my account console or Workspace I didnot get any option to enable or disable OBO Token

     

[MoJaMa]

So from your IoT device you don't have a way to use any of the Unified Authentication mechanisms?

https://docs.databricks.com/aws/en/dev-tools/auth/unified-auth

[Punit_Prajapati]

Hi Databricks Team,

Thanks for the response.

I reviewed the Unified Authentication documentation. From what I understand, the supported authentication methods are PAT, M2M (Service Principal OAuth), and U2M.

For my use case, external IoT devices are calling a Databricks App (FastAPI endpoint).

Both M2M and U2M require OAuth access tokens that expire after 3600 seconds. This means the devices must frequently re-authenticate, which is not practical for IoT devices deployed in the field.

I also tried using a User PAT to call the Databricks App endpoint. This always returns 401 Unauthorized, even though the user has access to the Databricks App and the same PAT works for Databricks REST APIs. It only fails for Databricks Apps endpoints.

So currently, OAuth tokens are too short-lived for IoT use cases, and PATs do not seem to work for Databricks Apps.

Could you please clarify the following:

Is PAT-based authentication officially supported for Databricks Apps, or is it not supported by design?

Is there any supported way to use long-lived or non-interactive authentication for external systems such as IoT or edge devices?

If neither PAT nor long-lived tokens are supported, what is the recommended architecture for this scenario, such as using a proxy service or token broker?