Hello,
I’m running into a Lakebase / Feature Store issue related to service principal authentication when trying to log or read from the Databricks Feature Store. Migrating from the legacy online tables.
Here’s the exact error:
psycopg2.OperationalError: connection to server at "instance.database.cloud.databricks.com" (private_ip), port 5432 failed: FATAL: Failed to get identity details for username: "user_uuid". Please reach out to Databricks support. From the serving enpoint.
Context
- Using: databricks.feature_engineering.FeatureEngineeringClient = 0.13.0
- Logging model via: fe.log_model(...)
- MLflow registry URI: databricks-uc
- Runtime: 14.3 LTS ML
- Unity Catalog and Lakebase are enabled
- The service principal exists in Unity Catalog and appears in system audit logs and permission tables
Observation
A minimal Lakebase training example (such as the default Feature Store + sklearn demo) runs without any issues the model is logged, registered, and the values are retrieved correctly from the lakebase for the model prediction when requesting from the serving endpoint.
The problem only appears when running a custom training pipeline.
What I’ve verified
- It appears in the system audit logs (system.access.audit) With the correct permissions accesses.
- It’s present in permissions tables (permissions, identities, etc.)
- Minimal Lakebase example (with sklearn + Feature Store) works fine
Is there a known limitation or configuration step required for service principals (serving endpoints) to authenticate against Lakebase ? Or some limitations when deploying custom models with the feature engineering lib related to the lakebase features retrieval ?
