Azure Databricks Default Package Repository with Azure Key Vault-backed Secret Scope

[kfadratek]

Hi,

I’m trying to configure Default Python Package Repository in an Azure Databricks workspace and would like to clarify whether the predefined secret scope can be backed by Azure Key Vault.

According to the documentation, Default Package Repository can use the predefined secret scope:

databricks-package-management

with the following keys:

pip-index-url
pip-extra-index-urls
pip-cert

The documentation shows an example using:

databricks secrets create-scope databricks-package-management followed by put-secret commands.

However, I cannot find an explicit statement saying that databricks-package-management must be a Databricks-backed secret scope.

The documentation also describes two types of secret scopes, including Azure Key Vault-backed scopes. An Azure Key Vault-backed scope is read-only from the Databricks side, so the secrets would be created directly in Azure Key Vault rather than using databricks secrets put-secret.

My question is:

Has anyone successfully configured Default Package Repository using databricks-package-management as an Azure Key Vault-backed secret scope?

I have already tested this configuration. The Key Vault-backed scope exists and contains pip-index-url, and Databricks can read the secret through the scope.

However, I cannot specify the secret scope in the Default Package Repositories UI, and I would like to understand whether Databricks automatically resolves the predefined databricks-package-management scope or whether this functionality specifically requires a Databricks-backed scope.

Has anyone tested this configuration and got it working?

Alternatively, does anyone have official information or a technical reference confirming that databricks-package-management must use a Databricks-backed scope and cannot use an Azure Key Vault-backed scope?

Thanks!

[ShamenParis]

Hi @kfadratek ,

Yes, an Azure Key Vault (AKV) backed scope will work, but Databricks has to resolve it automatically in the background.

The UI is built to write secrets. Since AKV scopes are read-only from the Databricks side, the UI won't let you map or manage it.

If you already created the AKV scope named exactly databricks-package-management and added the correct keys (like pip-index-url), the issue is likely just missing permissions.

Try these two steps:

1. Grant Databricks users read access: The cluster needs permission to read the scope when it boots up. Run this via the Databricks CLI:

   Bash

   databricks secrets put-acl databricks-package-management users READ

2. Check Azure permissions: Ensure the managed identity or service principal running your cluster actually has "Get" and "List" access directly inside the Azure Key Vault's access policies/RBAC.

Run that CLI command, restart your cluster, and it should automatically pick up the packages!