cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

create databricks scope by reading AWS secrets manager

thiagoawstest
Contributor

Hi, I have datbricks on AWS, I created some secrets in AWS Secrets Manger, I would need to create the scopes based on AWS secrets manager.

When I use Azure's Key Vault, when creating the scope, it uses the option -scope-backend-type AZURE_KEYVAULT, but I didn't find it for AWS.

How would I create a scope with which to read the secrets from AWS Secrets Manager?, or would it only be possible via Python code?

Thanks.

2 REPLIES 2

Yeshwanth
Honored Contributor
Honored Contributor

Hi @thiagoawstest 

Step 1: Create Secret Scope
You can create a secret scope using the Databricks REST API as shown below:

python
import requests
import json

# Define the endpoint and headers
url = "https://<databricks-instance>/api/2.0/secrets/scopes/create"
headers = {
    "Authorization": "Bearer <your-databricks-token>",
    "Content-Type": "application/json"
}

# Define the payload
payload = {
    "scope": "aws-secrets-scope",
    "initial_manage_principal": "users"
}

# Make the request
response = requests.post(url, headers=headers, data=json.dumps(payload))

if response.status_code == 200:
    print("Secret scope created successfully.")
else:
    print(f"Failed to create secret scope: {response.text}")

Is this what you are looking for? Please test it once before deploying it in the workload.

That wouldn't be it, I created secrets within the AWS secret manager, when I use Azure Key Vault when creating the scope in Databricks, I pass a parameter that reads the key vault, but for AWS I didn't find it to read the AWS secrets manager .

I wanted to understand if it is not supported, or would it only be via boto3?