Accessing secrets(secret scope) in pipeline yml file

prakharsachan
New Contributor III

How can I access secrets in pipeline yaml or directly in python script file?

szymon_dybczak
Esteemed Contributor III

Hi @prakharsachan ,

In Declarative Automation Bundles YAML (formerly known as Databricks Assets Bundles) you can only define secret scopes:

szymon_dybczak_0-1776102462918.png

If you want to read secrets from secret scope you can use dbutils in python script:

password = dbutils.secrets.get(scope = "<scope-name>", key = "<key-name>")

If you're talking more about devops pipeline you can also use databricks cli to read secret in following way:

databricks secrets get-secret <scope-name> <key-name> | jq -r .value | base64 --decode

 

More information you can find below:

Secret management | Databricks on AWS

View solution in original post

@szymon_dybczak 
But dbutils utility can be used only in notebooks , but cant be used in python .py files.

szymon_dybczak
Esteemed Contributor III

Hi @prakharsachan ,

If dbutils doesn't work for you then you can use Databricks Python SDK. Below you can find get_secret method which you can use to read secret from a secret scope:

w.secrets: Secret — Databricks SDK for Python beta documentation