amirabedhiafi
Contributor III

Hello @KhaturiabPreet  !

I like you analysis it made me do some reading this morning to freshen up my memory.

I checked the the doc and I can say that databricks_writer_<dbid> being unable to CONNECT does not look like an expected state because it is the system role used to create and manage synced tables and synced tables themselves are owned by that role. .

The doc does not say that users need to manually grant CONNECT to databricks_writer_<dbid> as part of synced table setup since its prerequisites concern the Lakebase project, UC source and permissions  create_database_objects_if_missing = true is also explicitly supported when creating the synced tablehttps://docs.databricks.com/aws/en/oltp/projects/sync-tables

Therefore, given that the database was created through the Lakebase UI and every synced table failed until the writer privilege was corrected this looks much more like provisioning or bug than a skipped setup step.

Another thingGRANT CONNECT explains and fixes the failure but I would not consider manual modification of the system role the ideal long term configuration procedure as standard PostgreSQL GRANT commands are the documented mechanism for Lakebase database privileges including database privileges. however databricks_writer_<dbid> is categorized as a system managed role used by internal services. 

so this:

GRANT CONNECT ON DATABASE <database_name>
TO databricks_writer_<dbid>;

is a technically correct remediation in my opinion as demonstrated by all three pipelines immediately succeeding but I would open a DBKS support case and ask them to confirm whether that database was incorrectly provisioned. 

There is another strong point supporting your case which is the documented behavior says changing the pipeline run as identity does not reassign synced table ownership so recreating under different user or SP identities would not fix an ACL problem on databricks_writer_<dbid>.

and yes checking effective CONNECT is a sensible temporary readiness check especially for older or existing databases but I would avoid parsing pg_database.datacl directly and yoy have has_database_privilege() specifically for checking database connection permissions.

For example:

SELECT
    has_database_privilege(
        'databricks_writer_<dbid>',
        '<database_name>',
        'CONNECT'
    ) AS writer_can_connect;

you could check both relevant database privileges:

SELECT
    has_database_privilege(
        'databricks_writer_<dbid>',
        '<database_name>',
        'CONNECT'
    ) AS writer_can_connect,
    has_database_privilege(
        'databricks_writer_<dbid>',
        '<database_name>',
        'CREATE'
    ) AS writer_can_create;

This is preferable to inspecting pg_database.datacl because it tests the effective permission including inherited or public privileges rather than requiring your automation to interpret PostgreSQL ACL strings.

I didn't find anything in the doc until now a documented lakebase management API that exposes effective PostgreSQL database grants directly.

The lakebase Postgres REST API is primarily for infrastructure or resource management and DBKS distinguishes that API from actual database access. So I think for an automated readiness test today executing has_database_privilege() over a PostgreSQL connection is the cleaner supported approach.

If this answer resolves your question, could you please mark it as “Accept as Solution”? It will help other users quickly find the correct fix.

Senior BI/Data Engineer | Microsoft MVP Data Platform | Microsoft MVP Power BI | Power BI Super User | C# Corner MVP