cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Secret scope with Azure RBAC

achistef
New Contributor III

Hello!

We have lots of Azure keyvaults that we use in our Azure Databricks workspaces. We have created secret scopes that are backed by the keyvaults. Azure supports two ways of authenticating to keyvaults:

- Access policies, which has been marked as legacy.

- Role-based access control (RBAC), which is a unified standard in all Azure services.

The Databricks secret scopes can only be defined using access policies. Given that Azure stirs towards RBAC, is there a plan to support RBAC for secret scopes?

1 ACCEPTED SOLUTION

Accepted Solutions

daniel_sahal
Esteemed Contributor

@achistef 

Actually, RBAC is supported for authentication for the secret scopes.

The thing is, when you setup the secret scope, Databricks is automatically assigning permissions through access policies. With RBAC - you'll need to grant the role on your own.

As a test:

1. I've created an Azure KeyVault with "Azure role-based access control" as a permission model.
2. Navigated to https://<databricks-instance>#secrets/createScope and created a secret scope
3. In Azure KeyVault IAM, added Key Vault Secrets User to the AzureDatabricks (fa5c679a-c02e-4f33-a397-7419315171b3) Application
4. Created a test secret and tried to access that from a notebook. Tada, it works.

View solution in original post

5 REPLIES 5

daniel_sahal
Esteemed Contributor

@achistef 

Actually, RBAC is supported for authentication for the secret scopes.

The thing is, when you setup the secret scope, Databricks is automatically assigning permissions through access policies. With RBAC - you'll need to grant the role on your own.

As a test:

1. I've created an Azure KeyVault with "Azure role-based access control" as a permission model.
2. Navigated to https://<databricks-instance>#secrets/createScope and created a secret scope
3. In Azure KeyVault IAM, added Key Vault Secrets User to the AzureDatabricks (fa5c679a-c02e-4f33-a397-7419315171b3) Application
4. Created a test secret and tried to access that from a notebook. Tada, it works.

 

Hi @achistef ,

As Daniel mentioned, RBAC is supported, but you should be aware of the consequences it entails.
For Datbricks to connect to Keyvault on RBAC we add AzureDatabricks Enterprise Application ID in RBAC, but this allows all the Databricks instances deployed in that tenant to have an access to that KeyVault.
You can read more in below discussion:

Allow only a specific Azure Databricks instance to connect to keyvault - Microsoft Q&A

And here is video how to configure Databricks and KeyVault using RBAC:

Unlocking Secrets in Azure Databricks with Azure Key Vault! ๐Ÿ—๏ธโœจ | Azure Databricks Tutorials (youtu...

 

@szymon_dybczak When using Access Policies, you're still adding the permissions to AzureDatabricks SP, so it's kinda the same issue as with RBAC. That's why I'm not a big fan of having secret scopes at all.

What's more, to even create a secret scope in Databricks, you need (i mean, a user who creates a secret scope) a Contributor or Owner role on the KeyVault, so that's a little bit of security that was added here.

@daniel_sahal  I'm not a fan of this solution either. The worst part is that neither Databricks nor Microsoft want to address this issue. And it's been known for years...
From security perspective, I think it's better to handle secrets by yourself, i.e writing custom library. That way you have much more granular control over who has access to what.

achistef
New Contributor III

That is very helpful, thank you for your answers.

FYI there is some outdated documentation about this topic 
https://learn.microsoft.com/en-us/azure/databricks/security/secrets/secret-scopes#configure-your-azu...

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group