cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Secrete management

Snoonan
Contributor

Hi all,

 

I am trying to use secrets to connect to my Azure storage account. I want to be able to read the data form the storage account using a pyspark notebook.

Has anyone experience setting up such a connection or has good documentation to do so?

I have come across Databricks managed secrets and Azure key vault managed secrets. I am not sure of the difference. Could anyone clarify the difference?

Thanks,

Sean

1 ACCEPTED SOLUTION

Accepted Solutions

DonatienTessier
Contributor

Hi Sean,

There are two ways to handle secret scopes:

  • databricks-backed scopes: scope is related to a workspace. You will have to handle the update of the secrets.
  • Azure Key Vault-backed scopes: scope is related to a Key Vault. It means than you configure the access to KV using a scope and then you will be able to access the secrets stored in the KV (if you configured properly the access first).

The security best practices is to use an Azure Key Vault-backed scopes. If there is some rotation policies activate, it will be handle.

Nevertheless, if you need to access to a storage account (in case of ADLS), it is better if you can use an access connector rather than using the access key, for example.

I hope it is clearer now 🙂

View solution in original post

3 REPLIES 3

AmanSehgal
Honored Contributor III

Have you tried using Azure key vault backed secret scope?

Secret scopes - Azure Databricks | Microsoft Learn

DonatienTessier
Contributor

Hi Sean,

There are two ways to handle secret scopes:

  • databricks-backed scopes: scope is related to a workspace. You will have to handle the update of the secrets.
  • Azure Key Vault-backed scopes: scope is related to a Key Vault. It means than you configure the access to KV using a scope and then you will be able to access the secrets stored in the KV (if you configured properly the access first).

The security best practices is to use an Azure Key Vault-backed scopes. If there is some rotation policies activate, it will be handle.

Nevertheless, if you need to access to a storage account (in case of ADLS), it is better if you can use an access connector rather than using the access key, for example.

I hope it is clearer now 🙂

Hi @DonatienTessier ,

This is very clear. Thank you!