We are facing an authorization issue when using serverless compute with ADLS Gen2 storage. Queries fail with:

AbfsRestOperationException: Operation failed: "This request is not authorized to perform this operation.", 403 AuthorizationFailure

Details:

• Environment: Azure Databricks with Unity Catalog enabled

• Storage: ADLS Gen2, external location configured

• Authentication: Unity Catalog storage credential using Service Principal (not SAS token)

• RBAC: Service Principal has Storage Blob Data Contributor role at the storage account level

• Behavior:

  • Queries succeed when using general purpose compute clusters

  • Queries fail with 403 when using serverless compute

Steps Tried:

1. Verified RBAC role assignment at both account and container level.

2. Confirmed external location is bound to the storage credential.

3. Granted usage on external location to UC groups.

4. Tested access via CLI with the same Service Principal — works fine.

Request for Help:

• Are there additional RBAC permissions or workspace entitlements required for serverless compute to access ADLS Gen2?

• Does serverless compute require a different configuration for Unity Catalog storage credentials compared to general compute?

Impact: We are currently using general compute clusters as a workaround, but need serverless compute enabled for production workloads.