PERMISSION_DENIED: Request for user delegation key is not authorized.

[hietpas]

I am attempting to copy files from an Azure Storage container using an Azure Databricks Volume. When attempting to list files using dbutils.fs.ls('/Volumes/myCatalog/mySchema/myVolume' I get the following error:
ExecutionError: (com.databricks.sql.managedcatalog.acl.UnauthorizedAccessException) PERMISSION_DENIED: Request for user delegation key is not authorized. Details: None

Note that this differs from previous error messages where a user has insufficient grants. I cannot find any other references to "delegation key" that address this.

The Volume is based on an External Location pointing to the ADLS container. I am using an access connector for Databricks identity, which has Storage Blob Data Reader role on the container. I granted READ VOLUME on the volume. I granted USE SCHEMA and USE CATALOG on the catalog containing the schema / volume. I granted BROWSE and READ FILES on the External Location. Within the catalog explore, I can test the External Location connection and confirm read access and files are listed. The Volume also displays the files. Any idea why the "delegation" might fail?

I previously tested a similar scenario and it worked.

[szymon_dybczak · Accepted solution]

Hi @hietpas ,

I think your access connector doesn't have sufficient permission to storage account. Check below documentation entry. Try to grant Storage Blob Data Contributor role for your connector.

[hietpas]

@szymon_dybczak Can you provide a link to the documentation you noted? I confirmed this with my own testing, that the Storage Blob Delegator role must be at the ADLS account-level, and Storage Blob Data Reader can then be applied at the container-level. However, I couldn't find any documentation to support this.

[szymon_dybczak]

Hi @hietpas ,

Sure, here it is:

Use Azure managed identities in Unity Catalog to access storage - Azure Databricks | Microsoft Learn