topic Re: Accessing secrets(secret scope) in pipeline yml file in Data Engineering

Accessing secrets(secret scope) in pipeline yml file

prakharsachan — Mon, 13 Apr 2026 17:35:51 GMT

How can I access secrets in pipeline yaml or directly in python script file?

szymon_dybczak — Mon, 13 Apr 2026 17:49:53 GMT

In Declarative Automation Bundles YAML (formerly known as Databricks Assets Bundles) you can only define secret scopes:

If you want to read secrets from secret scope you can use dbutils in python script:

password = dbutils.secrets.get(scope = "<scope-name>", key = "<key-name>")

If you're talking more about devops pipeline you can also use databricks cli to read secret in following way:

databricks secrets get-secret <scope-name> <key-name> | jq -r .value | base64 --decode

More information you can find below:

prakharsachan — Tue, 14 Apr 2026 05:38:30 GMT

@szymon_dybczak
But dbutils utility can be used only in notebooks , but cant be used in python .py files.

szymon_dybczak — Tue, 14 Apr 2026 07:25:43 GMT

If dbutils doesn't work for you then you can use Databricks Python SDK. Below you can find get_secret method which you can use to read secret from a secret scope:

w.secrets: Secret — Databricks SDK for Python beta documentation