Secrete management

Snoonan
Contributor

‎05-09-2024 02:00 AM

Hi all,

 

I am trying to use secrets to connect to my Azure storage account. I want to be able to read the data form the storage account using a pyspark notebook.

Has anyone experience setting up such a connection or has good documentation to do so?

I have come across Databricks managed secrets and Azure key vault managed secrets. I am not sure of the difference. Could anyone clarify the difference?

Thanks,

Sean

0 Kudos

AmanSehgal
Honored Contributor III

‎05-09-2024 05:49 AM

Have you tried using Azure key vault backed secret scope?

Secret scopes - Azure Databricks | Microsoft Learn

0 Kudos

DonatienTessier
Databricks Partner

‎05-13-2024 01:36 AM

Hi Sean,

There are two ways to handle secret scopes:

  • databricks-backed scopes: scope is related to a workspace. You will have to handle the update of the secrets.
  • Azure Key Vault-backed scopes: scope is related to a Key Vault. It means than you configure the access to KV using a scope and then you will be able to access the secrets stored in the KV (if you configured properly the access first).

The security best practices is to use an Azure Key Vault-backed scopes. If there is some rotation policies activate, it will be handle.

Nevertheless, if you need to access to a storage account (in case of ADLS), it is better if you can use an access connector rather than using the access key, for example.

I hope it is clearer now 🙂

View solution in original post

0 Kudos

Snoonan
Contributor
In response to DonatienTessier

‎05-22-2024 04:03 AM

Hi @DonatienTessier ,

This is very clear. Thank you!

0 Kudos