10-10-2018 11:52 AM
I am trying to set retrieve a secret from Azure Key Vault as follows:
sqlPassword = dbutils.secrets.get(scope = "Admin", key = "SqlPassword")
The scope has been created correctly, but I receive the following error message:
com.databricks.common.client.DatabricksServiceException: INVALID_STATE: Databricks could not access the keyvault: https://AzureKeyVaultName.vault.azure.net/.
I assume I need to add Databricks in Azure Key Vaults "Access Policies" however I cannot find any information online on how to do this. Any ideas?
06-10-2019 10:48 PM
FYI I found the answer to my problem and posted about it here on SO to share: https://stackoverflow.com/questions/56537214/creating-a-secret-scope-in-databricks-backed-by-azure-k...
It was because I was using a user in Databricks that did not have rights in Azure AD to make a service principal.
01-09-2019 04:33 AM
01-19-2019 09:23 PM
You can follow the steps mentioned at:
https://medium.com/@cprosenjit/azure-databricks-with-azure-key-vaults-c00df6548222
06-10-2019 08:44 PM
This did not work for me. I have the same error - including misspelling -
"Internal error happened while granting read/list permission to Databricks ervice principal to KeyVault: XYZ"
I'm assuming Databricks is using a default service principal in Azure AD to communicate with KeyVault but I don't have access to AD and I can't find the Databricks principal name.
03-05-2019 01:27 PM
Hey did you solve the issue?
06-10-2019 10:48 PM
FYI I found the answer to my problem and posted about it here on SO to share: https://stackoverflow.com/questions/56537214/creating-a-secret-scope-in-databricks-backed-by-azure-k...
It was because I was using a user in Databricks that did not have rights in Azure AD to make a service principal.
08-10-2020 09:15 PM
Sometimes turning it off and on again is underrated, so I gave up finding the problem, deleted it and re-created the scope - worked a breeze!
Mine seems like it was something silly, I was able to set up my vault but got the same issue when trying to use it 1hr later - even when logged in as myself, an admin of the workspace. Whenever I created it before with the exact same process (via a pipeline), it worked, just not this time... so no problem ID here, just a solution 🙂Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
Click here to register and join today!
Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.