Calling get_token() on a Unity Catalog service credential fails for any scope using the api:// App ID URI format. Only https://-scheme resource scopes succeed. The same api:// scopes work correctly with a service principal.
Reproduction
credential = dbutils.credentials.getServiceCredentialsProvider(
"managed-identity"
)
scopes = [
"https://vault.azure.net/.default",
"api://<domain>/.default",
"api://<domain>",
"api://<domain>/",
]
for scope in scopes:
try:
result = credential.get_token(scope)
print(scope, "OK; expires:", result.expires_on)
except Exception as error:
print(scope, type(error).__name__, str(error))Result
The first scope succeeds. Every api:// variant fails with:
Error while retrieving temporary credentials: '<scope>' is not a valid URI
Substituting the app registration's Application (client) ID GUID — both as api://<client-id>/.default and as a bare <client-id>/.default — fails identically.
This isolates the failure to scope-string validation in the temporary-credentials path, not to the credential, the managed identity, or its permissions — the same credential object returns a valid token for the https:// scope in the same loop.
Control test: the same scope succeeds via ClientSecretCredential
On the same compute, in the same notebook, the identical api:// scope returns a valid token when requested through a service principal with azure.identity instead of the service credential provider:
from azure.identity import ClientSecretCredential
client_id = dbutils.secrets.get(scope="keyvault", key="client_id")
client_secret = dbutils.secrets.get(scope="keyvault", key="client_secret")
tenant_id = dbutils.secrets.get(scope="keyvault", key="tenant_id")
credential = ClientSecretCredential(
tenant_id=tenant_id,
client_id=client_id,
client_secret=client_secret
)
scopes = ["api://<domain>/.default"]
for scope in scopes:
try:
result = credential.get_token(scope)
print(scope, "OK; expires:", result.expires_on)
except Exception as error:
print(scope, type(error).__name__, str(error))Result
api://<domain>/.default returns a token successfully.
This confirms the scope string is valid and that the app registration is correctly configured. The rejection originates solely in the Databricks temporary-credentials layer.
Environments tested
- Serverless compute
- Classic compute, DBR 17.3 LTS
Identical behavior on both.
Business impact
Our security policy mandates managed identities. Unity Catalog service credentials are therefore our only supported path for machine-to-machine authentication from Databricks to internal APIs. This validation blocks that path entirely for any internal API using the default api:// identifier URI.
The ClientSecretCredential approach shown above works, but is not a viable workaround for us: it requires storing and rotating a client secret, which is precisely what the managed-identity mandate exists to avoid.
Questions
- Is this a known limitation, or a bug in scope validation for temporary service credentials?
- If intentional, where is the accepted resource-URI format documented? We found no mention of a scheme restriction in the service credentials documentation.
