The error you’re encountering —
psycopg2.OperationalError: FATAL: Failed to get identity details for username: "user_uuid" — typically arises from an OAuth identity mismatch or invalid token scope when a Databricks service principal is used to authenticate against a Lakebase Feature Store database instance.
Here’s how to interpret and resolve this issue.
Root Cause
Lakebase requires OAuth-based identity validation when authenticating to its Postgres-compatible endpoint. Each connection validates that the token’s Databricks identity (service principal or user) matches the security label (Postgres role) configured for the Lakebase instance.
Your custom pipeline likely triggers the mismatch because:
-
The service principal token used by the serving endpoint is not correctly scoped to the Lakebase database’s workspace identity.
-
The token may have been generated from a different workspace, or the identity context (client_id / secret) used by the FeatureEngineering library is not resolving to the role assigned to the Lakebase instance.
-
Minimal examples succeed because they run under a user context that matches the workspace’s identity, while custom pipelines use machine-to-machine (M2M) credentials.
Authentication Requirements
To resolve the issue, ensure these conditions are met as per the official Databricks Lakebase documentation :
-
Tokens must be workspace-scoped
OAuth tokens are valid only within the workspace owning the Lakebase database. Cross-workspace token usage is unsupported.
-
Correct Service Principal Configuration
The service principal must:
-
Be explicitly assigned to the workspace hosting the database (databricks accounts assign-service-principal-to-workspace).
-
Have the necessary Unity Catalog permissions for both the feature table and the Lakebase database instance.
-
Generate the Token in a Machine-to-Machine Flow
You must mint an OAuth token specifically for the database context:
databricks database generate-database-credential \
--request-id $(uuidgen) \
--json '{"instance_names": ["<db-instance-name>"]}'
This returns a JSON payload such as:
{
"expiration_time": "2025-10-24T14:15:22Z",
"token": "DATABRICKS_OAUTH_TOKEN"
}
Then, use the token as your Postgres password in your feature-serving endpoint configuration.
-
Verify Token Identity
Use the following command to ensure the identity bound to the token matches the expected role:
from databricks.sdk import WorkspaceClient
w = WorkspaceClient()
print(w.current_user.me().user_name)
The output must match the service principal configured in Unity Catalog and granted access to the Feature Store database.
-
Ensure SSL is Enabled
Token-based login requires SSL (Lakebase enforces authenticated, encrypted sessions only).
Known Limitations
From Databricks documentation and the current Feature Store runtime behavior :
-
Cross-workspace feature retrieval is unsupported.
-
Only Unity Catalog–managed feature tables can be accessed through Lakebase.
-
The service principal identity must exist and be directly linked in the permissions tables (it will not inherit access transitively from groups).
-
Tokens expire every hour, but connections remain active until closed — rotation is required for long-running processes.
Recommended Fix Path
-
Regenerate a M2M OAuth token using the same workspace where the Lakebase instance resides.
-
Confirm the service principal appears in:
-
system.access.audit (successful OAuth handshake)
-
information_schema.role_authorization_descriptors
-
Update your custom model code to pass the correct token to FeatureEngineeringClient():
fe = FeatureEngineeringClient(
host="https://<your-instance>.cloud.databricks.com",
client_id="<service_principal_id>",
client_secret="<service_principal_secret>"
)
-
Re-log the model with fe.log_model(...) once the identity context matches.
If the issue persists, Databricks support recommends checking if your workspace has Public Preview access for Lakebase M2M OAuth, as it’s still rolling out progressively by region.
