cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Synced table pipeline fails with permission denied for database

KhaturiabPreet
Visitor

Environment: Azure Databricks, Lakebase Autoscaling project. Database created through the Lakebase UI by the project owner (audit log shows createProjectBranchDatabaseUI with a browser user agent). Synced tables created via the databricks_postgres_synced_table Terraform resource, provider 1.122.0, scheduling_policy = TRIGGERED, create_database_objects_if_missing = true.

Symptom: Synced tables are created successfully in Unity Catalog and Terraform reports success, but every pipeline's initial load fails and the Postgres tables are never created. Status is SYNCED_TABLE_OFFLINE_FAILED. The surfaced message is only:

[SYNCED_TABLE_ERROR] Synced table pipeline failure. Instance: <endpoint>. Please contact Databricks support.

The real cause is two levels deep in the pipeline events:

ERROR: permission denied for database "<mydb>"

What we ruled out: we recreated the synced tables under two different creator identities, both of which already had CONNECT and CREATE on the database. Both failed identically. That matches the documented behaviour that a synced table is owned by databricks_writer_<dbid> rather than by the creator, and that changing the pipeline's Run as identity doesn't reassign it. So the creator's privileges are not the issue.

Diagnosis: pg_database.datacl showed the database's own writer role missing CONNECT, while both reader roles had it:

databricks_writer_<oid> = C -- CREATE only, no CONNECT
databricks_reader_<oid> = c
databricks_reader_<other> = c
databricks_superuser = C*T*c*

SELECT oid FROM pg_database WHERE datname = '<mydb>' confirmed <oid> is this database, so that is the correct writer role for it.

No synced table had ever succeeded in this database — the only Databricks-owned table was __db_system.unity_catalog_registration_status.

Fix: granting the missing privilege, as a databricks_superuser member:

GRANT CONNECT ON DATABASE <mydb> TO databricks_writer_<oid>;

All three pipelines completed on the next trigger, with no other change, and row counts matched the source exactly.

Questions:

1. Is it expected that a UI-created Lakebase database can have CREATE but not CONNECT on its own databricks_writer_<dbid> role? Or does this indicate a provisioning bug?
2. Is granting CONNECT to that role the supported fix, or is there a setup step we skipped? The docs warn that modifying system roles can affect instance behaviour, so I would rather not carry a workaround indefinitely.
3. Should this be a routine readiness check before adding a synced table to an existing database — particularly one created some time ago — and if so, is there a supported API for it rather than querying pg_database.datacl directly?

0 REPLIES 0