Azure Databricks Unity Catalog - cannot access managed volume in notebook

AlbertWang
Valued Contributor

We have set up Azure Databricks with Unity Catalog (Metastore).

  • Used Managed Identity (Databricks Access Connector) for connection from workspace(s) to ADLS Gen2
  • ADLS Gen2 storage account has Storage Blob Data Contributor and Storage Queue Data Contributor at the storage account level granted to the Databricks Access Connector
  • Used Storage Credentials and External Location
  • I am a Databricks account admin and the metastore admin
  • Created catalog and mapped to a External Location (ASDL Gen2 container)
  • Created schema and mapped to a External Location (ASDL Gen2 container)
  • Created a volume
  • I can uploaded files to the volume on Databricks Portal
  • I can browse the files in the volume on Databricks Portal
  • Everything (workspaces, ADLS Gen2, etc) is in the same Azure region

When I run `

dbutils.fs.ls("/Volumes/catalog1/schema1/volume1")`, I got error "Operation failed: "This request is not authorized to perform this operation.", 403, GET".
 
Any help will be appreciated!