Databricks AWS Secrets Manager access
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-03-2023 04:50 PM - edited 10-03-2023 04:58 PM
I have a workspace deployed in AWS and need to read some secrets from AWS Secrets Manager in my notebook. I'm aware that there is no default process similar to Azure Key Vault, however I know that we can try to access it using boto3, but I'm stuck at the authentication process. I'm not allowed by the company to create key values for it, so it needs to be done with roles. I've been trying to follow the Databricks documentation to use instance profiles in the cluster (https://docs.databricks.com/en/aws/iam/instance-profile-tutorial.html) but it's specific for s3 and not necessarily for AWS secrets manager. If someone have done it before and could share how is the appropriated way, it would be very appreciated!