Mounting adls gen2 from databricks RBAC issue

[keer1392]

I am trying to mount my Gen 2 storage account in databricks. I have added permission of Storage blob data contributor to the storage account. But I am getting below error:
Invalid permissions on the specified KeyVault https://kvmigrationnew.vault.azure.net/. Wrapped Message: Status code 403, {"error":{"code":"Forbidden","message":"Caller is not authorized to perform action on resource.\r\nIf role assignments, deny assignments or role definitions were changed recently, please observe propagation time.\r\nCaller: name=AzureDatabricks;appid=123abcdbhjdllajddaddd;iss=https://sts.windows.net/984433r41f5ssadfgfsf/\r\nAction: 'Microsoft.KeyVault/vaults/secrets/getSecret/action'\r\nResource: '/subscriptions/xxxx-xxx-xxx/resourcegroups/rsggen2demo/providers/microsoft.keyvault/vaults/kvmigrationnew/secrets/clientid'\r\nAssignment: (not found)\r\nDecisionReason: 'DeniedWithNoValidRBAC' \r\nVault: kvmigrationnew;location=eastus2\r\n","innererror":{"code":"ForbiddenByRbac"}}}
------------------------------------
Configuration
---------------
Resource group: rsggen2demo
Storage acc name: stggen2demo
Keyvault: kvmigrationnew
DB Scope: gen2mig

Code:
adlsAccountName = "stggen2demo"
adlsContainerName = "output"
adlsFolderName = "schemas_new"
mountPoint = "/mnt/schemas_new"

# Application (Client) ID
applicationId = dbutils.secrets.get(scope="gen2mig",key="ClientID")

# Application (Client) Secret Key
authenticationKey = dbutils.secrets.get(scope="gen2mig",key="ClientSecret")

# Directory (Tenant) ID
tenandId = dbutils.secrets.get(scope="gen2mig",key="TenantID")

endpoint = "https://login.microsoftonline.com/" + tenandId + "/oauth2/token"
source = "abfss://" + adlsContainerName + "@" + adlsAccountName + ".dfs.core.windows.net/" + adlsFolderName

# Connecting using Service Principal secrets and OAuth
configs = {"fs.azure.account.auth.type": "OAuth",
"fs.azure.account.oauth.provider.type": "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider",
"fs.azure.account.oauth2.client.id": applicationId,
"fs.azure.account.oauth2.client.secret": authenticationKey,
"fs.azure.account.oauth2.client.endpoint": endpoint}

# Mount ADLS Storage to DBFS only if the directory is not already mounted
if not any(mount.mountPoint == mountPoint for mount in dbutils.fs.mounts()):
dbutils.fs.mount(
source = source,
mount_point = mountPoint,
extra_configs = configs)

[Kaniz_Fatma]

Hi @keer1392, The error you are encountering is due to the Azure Databricks application not having the appropriate permissions to access the secrets in the Azure Key Vault.

The error message indicates that the Azure Databricks application is not authorized to perform the ’Microsoft.KeyVault/vaults/secrets/getSecret/action’ on the KeyVault.

To resolve this issue, you must grant the Azure Databricks application the necessary permissions on the Azure Key Vault. Specifically, you need to add an access policy in the Key Vault for the Azure Databricks application and grant it the ’Get’ and ’List’ permissions under ’Secret permissions’.

Unfortunately, I cannot provide the exact commands to perform these actions as they are typically done via the Azure portal or Azure CLI, not within Databricks.

When using the mount commands, remember to replace the placeholders with your actual values. For example, replace <container-name> with the name of your container, <storage-account-name> with the name of your storage account and <mount-name> with the name of your mount point.

Here is an example of how you might mount your storage:

python
configs = { 
    "fs.azure.account.auth.type": "CustomAccessToken", 
    "fs.azure.account.custom.token.provider.class": spark.conf.get("spark.databricks.passthrough.adls.gen2.tokenProviderClassName") 
}

dbutils.fs.mount( 
    source = "abfss://<container-name>@<storage-account-name>.dfs.core.windows.net/", 
    mount_point = "/mnt/<mount-name>", 
    extra_configs = configs)

Please note that you must replace <container-name>, <storage-account-name>, and <mount-name> with your actual values.