I am attempting to copy files from an Azure Storage container using an Azure Databricks Volume. When attempting to list files using dbutils.fs.ls('/Volumes/myCatalog/mySchema/myVolume' I get the following error:
ExecutionError: (com.databricks.sql.managedcatalog.acl.UnauthorizedAccessException) PERMISSION_DENIED: Request for user delegation key is not authorized. Details: None
Note that this differs from previous error messages where a user has insufficient grants. I cannot find any other references to "delegation key" that address this.
The Volume is based on an External Location pointing to the ADLS container. I am using an access connector for Databricks identity, which has Storage Blob Data Reader role on the container. I granted READ VOLUME on the volume. I granted USE SCHEMA and USE CATALOG on the catalog containing the schema / volume. I granted BROWSE and READ FILES on the External Location. Within the catalog explore, I can test the External Location connection and confirm read access and files are listed. The Volume also displays the files. Any idea why the "delegation" might fail?
I previously tested a similar scenario and it worked.
