topic Re: Databricks Key Vault Secret - Is it available in Databricks on AWS? in Data Engineering

Databricks Key Vault Secret - Is it available in Databricks on AWS?

Ramana — Thu, 05 Jun 2025 18:23:24 GMT

@Hubert-Dudek, I see your post regarding Key Value Secret handling via UI for Databricks on Azure.

Is this feature available for Databricks on AWS as well?

#Secrets #Scopes #DatabricksOnAWS

Thanks

Ramana

Re: Databricks Key Vault Secret - Is it available in Databricks on AWS?

lingareddy_Alva — Thu, 05 Jun 2025 19:52:35 GMT

Hi @Ramana

Yes, the Key-Value secret handling feature is available for Databricks on AWS as well! The Databricks Secret Management functionality works across all cloud platforms (AWS, Azure, and GCP) with very similar capabilities.

Databricks Secrets on AWS
Available Secret Scopes:
1. Databricks-backed scopes - Managed by Databricks itself
2. AWS Key Vault-backed scopes - Integrated with AWS Secrets Manager or AWS Systems Manager Parameter Store

This works the same way on AWS as it does on Azure - you can create, manage, and view secret scopes through the web interface.
The functionality is essentially identical across cloud platforms, with the main differences being in the backend integration options (AWS services vs Azure services vs GCP services).

Re: Databricks Key Vault Secret - Is it available in Databricks on AWS?

Ramana — Thu, 05 Jun 2025 20:05:50 GMT

Is there a reference article from Databricks or anyone talking about it?

That would be helpful to share with my team.

Re: Databricks Key Vault Secret - Is it available in Databricks on AWS?

lingareddy_Alva — Thu, 05 Jun 2025 20:39:51 GMT

Go through documents below.

Databricks Secret Management Guide:
https://docs.databricks.com/security/secrets/index.html

AWS-Specific Documentation:
Store secrets in AWS Secrets Manager:
https://docs.databricks.com/en/security/secrets/aws-secrets-manager.html

Secret scope backed by AWS Secrets Manager: This is typically covered within the main secrets documentation

Databricks CLI for Secrets:
Databricks CLI Secrets Commands: https://docs.databricks.com/dev-tools/cli/secrets-cli.html

CLI Installation and Setup: https://docs.databricks.com/dev-tools/cli/index.html

Re: Databricks Key Vault Secret - Is it available in Databricks on AWS?

Ramana — Thu, 05 Jun 2025 20:55:52 GMT

Currently, we manage Scopes and Secrets via CLI, API, and Terraform.

Today, I read an article about UC to Azure Key-Vault Integration written by @Hubert-Dudek, I thought that there would be a similar option for AWS as well. But it looks like it is not available for Databricks on AWS.

Here is the one I am referring to: https://databrickster.medium.com/unity-catalog-to-azure-key-vault-no-more-dbutils-secrets-0a8248af46df

FYI: https://docs.databricks.com/en/security/secrets/aws-secrets-manager.html This link is broken.

We also use AWS Secret Manager, but I don't think there is a tight integration between AWS and Databricks like Azure and Databricks for Key Vault.

If you know or see any article related to the one I provided, please share. That would be appreciated.

Re: Databricks Key Vault Secret - Is it available in Databricks on AWS?

lingareddy_Alva — Thu, 05 Jun 2025 21:15:36 GMT

I agree. Sure, I will share if I come across. Thanks.

Re: Databricks Key Vault Secret - Is it available in Databricks on AWS?

ceceliac — Mon, 04 Aug 2025 20:52:12 GMT

Hi, do you have any update on this topic? We are looking for the same thing. We are using Databricks (UC) on AWS and have a developer group that wants to implement 90-day rotation for an AWS IAM secret using Secrets Manager. I do not see anything in the documentation indicating that this is supported / possible.

Re: Databricks Key Vault Secret - Is it available in Databricks on AWS?

Ramana — Mon, 08 Sep 2025 17:06:37 GMT

I haven't heard anything yet.