Lakebase Discussions topics

Lakebase Postgre updating Delta Table.

rkhbo3003 — Wed, 02 Sep 2026 01:44:54 GMT

I am using Postgre for OLTP processing for POS application.Lag is reduced a lot, however when there is updation on Postgre table, I need to sync back to delta table. There is one way from delta table sync table (read only). Any design pattern sas to keep delta table and postgre table in sync.

Group-based auth with Lakebase Data API

Malthe — Tue, 06 Oct 2026 09:15:29 GMT

While Lakebase supports group-based authentication, this doesn't seem to have carried over to Lakebase Data API.

Here, we must assign on the individual user or service principal,

Create principal
Setup authenticator access
Schema- and table grants

This means that it becomes rather a big administrative burden to set up for multiple users or service principals, as well as disorganized.

Will this be supported in the future?

Lakebase Data API (GCP) returns jwk not found for valid service principal tokens

eduardostzouze — Thu, 24 Sep 2026 15:22:46 GMT

Workspace on GCP us-central1, Lakebase Autoscaling (Beta). The Data API returns 400 {"message":"jwk not found"} for every request.

Setup follows the docs: role created with databricks_create_role('<sp-uuid>', 'SERVICE_PRINCIPAL'), GRANT "<sp-uuid>" TO authenticator (pg_has_role returns true), USAGE/SELECT granted, schema exposed, schema cache refreshed, Data API disabled and re-enabled. Reproduced in a brand new project with a plain table in public.

Tokens tested: M2M workspace token (/oidc/v1/token, scope all-apis) and database credential (/api/2.0/postgres/credentials). Both have iss = workspace, aud = workspace ID, not expired, kid = _iSisQ. That kid is present in the workspace's published jwks_uri (us-central1.gcp.databricks.com/oidc/jwks.json ).

A psql/JDBC connection with the same service principal and the same database credential works fine.

Has anyone seen this on GCP? Is there a known issue with the Data API resolving JWKS on GCP workspaces?

Lakebase synced table doesn’t recognize Auto CDF on a SDP materialized view

batch_bender — Thu, 13 Aug 2026 14:34:11 GMT

I’m trying to create a triggered Lakebase synced table from a SDP-created materialized view.

The source MV uses Automatic CDF: row tracking is enabled and legacy CDF (delta.enableChangeDataFeed) is disabled. The Automatic CDF workspace preview is enabled (says GA now)

The producer pipeline is configured with the PREVIEW channel and pipelines.externalMetadata.enabled=true. I recreated and reran the pipeline successfully.

However, synced-table creation still says that Change Data Feed is required and suggests running ALTER TABLE ... SET TBLPROPERTIES (delta.enableChangeDataFeed=true). That would enable legacy CDF, and ALTER TABLE also cannot be used because the source is a materialized view.

Does Lakebase synced-table creation support Automatic CDF for pipeline-created materialized views? If so, how should pipeline_channel=PREVIEW be configured for the managed sync pipeline?

Lakebase Postgres branch stuck "disabled" after auto-archive → unarchive (Public Preview)

jeremiasInetum — Mon, 07 Sep 2026 14:18:19 GMT

Hi all,

Running into what looks like a platform bug with Lakebase Postgres (Public Preview) and hoping a community engineer can help, since my Databricks support case (#01009170) was closed as out of entitlement (personal account, no support contract) before anyone could look at the actual issue.

Setup: project `ontobricks-demo`, branch `production` (default), endpoint `primary`. The branch auto-archived after about two weeks of inactivity ("Automatically archived ... due to inactivity" on the Branch overview page).

What I did: opened the branch, saw the "This branch is archived. Connecting to the branch will unarchive it." banner, and used the branch-level "Connect" button, which generated a working-looking connection string and showed the compute as "primary • Idle". I also enabled branch protection ("Protect").

What's still broken, over an hour later and after multiple retries/reloads:

- The Computes tab still shows `primary` as `SUSPENDED`.
- Monitoring → System operations shows "Timeline unarchive" as `OK`, but no "Start compute" operation ever follows it (compare to the branch's initial creation, where "Create timeline" was immediately followed by "Start compute").
- Every real Postgres connection — from my own application, and from the Databricks SQL Editor itself — is rejected with:

*** ERROR: The endpoint has been disabled. Enable it using the API and retry. ***

- In the SQL Editor, the compute selector shows "No computes" collapsed, and even though it lists `primary ● Idle` when expanded, selecting it doesn't let me actually run a query.
- A separate feature on the same project, Data API, independently returns "Temporarily Unavailable."

So it looks like the branch-level unarchive succeeded, but the compute endpoint itself never actually resumed, and something in that project may be in a stuck/inconsistent state (control plane says "Idle", data plane refuses connections).

Has anyone seen this, or know the right way to force a full compute resume after an archive → unarchive cycle? I'd rather not delete/recreate the `primary` compute blind, since I'm not sure whether that's safe here or would just reproduce the same stuck state.

Thanks!

Lakebase Data API – First Request Times Out After 8 Seconds When Compute Is Offline

KeatOoi — Thu, 03 Sep 2026 10:55:15 GMT

Hi Everyone,

I’m using the Databricks Lakebase Data API and have noticed that the first API call consistently times out after approximately 8 seconds when the Lakebase compute is offline.

The first request triggers the compute to start, and the compute starts successfully, but the API request is cancelled after ~8 seconds. Subsequent requests work normally.

The error is:

{
  "code": "57014",
  "message": "canceling statement due to statement timeout",
  "details": null,
  "hint": null
}

I checked the authenticator role and noticed that the statement_timeout is configured to 8 seconds:

SELECT rolname, rolconfig
FROM pg_roles
WHERE rolname = 'authenticator';

Result:

authenticator | {statement_timeout=8s}

I suspect this statement_timeout may be causing the first request to be cancelled while the compute is starting.

I have permission to alter role, and I tried increasing it to 60 seconds, but received:

ERROR: permission denied to alter role (SQLSTATE 42501)

Has anyone experienced a similar issue? Is there a way to increase or configure the statement_timeout for the Lakebase Data API authenticator role?

Thank you.

How to prevent users from creating Lakebase compute?

charl-p-botha — Thu, 18 Jun 2026 13:35:32 GMT

Dear community,

According to [1] and other sources, all workspace users are assigned `CAN_CREATE` on lakebase projects, and this permission "can't be revoked".

The problem is that such a project comes with by default a 8 - 16 CU lakebase compute instance (Scale-to-zero is enabled, but with a 24-hour idle timeout, any connection or query immediately resumes it, and it has a non-zero minimum (always-on baseline)), which means that anyone of our workspace(s) users is able to rack up a sizeable bill by accident. (the moment you create the project, the compute starts running).

After an in-depth exploration of all documentation and also the latest databricks cli, I have not been able to find any way to disable this regrettable default.

Please suggest a way whereby workspace users can be prevented from creating lakebase projects? We DO want to use lakebase for a number of our products, but we definitely also need to be able to specify who is able to create / use and who is not. (fully disabling the feature via support ticket as suggested in this forum post [2] would not work)

It would be far preferable to have it as an entitlement, or even connected to an existing entitlement (the aptly titled "Allow unrestricted cluster creation" could work), or first prize would be a revokable / assignable privilege. As it stands, there are no usable levers, which is highly uncharacteristic of Databricks products.

Please help.

Kind regards,
Charl Botha, Stone Three

[1] https://learn.microsoft.com/en-us/azure/databricks/oltp/projects/grant-permissions-programmatically
[2] https://community.databricks.com/t5/lakebase-discussions/disable-lakebase-and-model-serving-foundation-models-at-account/m-p/148792

Lakebase Data API private access with Public Network Access disabled

POCUSER — Fri, 29 May 2026 07:58:24 GMT

We are testing Azure Databricks Lakebase Autoscaling with Public Network Access disabled and standard inbound Private Link enabled.

The workspace UI works privately through VPN, but the Lakebase Data API hostname still resolves to a public IP and returns:

HTTP 403: Public access is not allowed for workspace

According to the docs, Service Direct Private Link is not required when using only the Data API.

Has anyone successfully used Lakebase Data API privately with Public Network Access disabled?

If yes, what DNS or Private Link configuration is required? Should the Data API hostname resolve through the workspace inbound Private Link, or is another private endpoint/DNS setup needed?

Thanks!

Inquiry regarding Query History and Audit Logs for Databricks Lakebase

POCUSER — Mon, 25 May 2026 05:13:44 GMT

We are using Lakebase Data API (HTTP Endpoint) to execute queries and need to verify the audit log capabilities for compliance. Could you please clarify:

Query Text Logging: Does Databricks capture the full SQL statement text and the actual user identity for every request sent via the Lakebase Data API?
Log Location: Where are these Data API history logs stored (e.g., in system.query.history, Audit Logs in Storage Bucket, or via Unity Catalog)?
Error Logging: Are failed queries (Query Errors) and their detailed error messages recorded in these logs as well?

Lakebase login via REST for a service principal

_Lilith — Fri, 24 Apr 2026 10:40:37 GMT

Hi all,

I’m trying to set up REST-based communication between my Lakebase and a REST-client.

I’m following the documentation “Connecting to Lakebase via REST using a service principal” to obtain a workspace-level token. After that, I use the Lakebase Data API to authenticate against Lakebase and create/configure my user.

However, I’m running into a 403 error with the message:

Permission denied to set role {SP client ID}

My assumption is that this happens because I can’t grant the authenticator role to a service principal.

Does this mean that using user-based OAuth or username/password authentication is the only way to access Lakebase via REST? Neither of these options seems very secure to me.
Or am I missing a supported approach for service principals here?

Thanks in advance!

"Get Started with Lakebase" Course

MrSky — Thu, 23 Apr 2026 17:19:32 GMT

Greetings,

I am following course " Get Started with Lakebase Course". I got access to "Databricks Academy Labs" but I could not find documentation related to "Get Started with Lakebase Course". I appreciate any help.

Thanks,

Sri

Lakebase in an enterprise setup

Lode — Thu, 09 Apr 2026 08:08:51 GMT

Hi,

Has anyone managed to get the new Lakebase autoscaling fully working in an enterprise Azure setup?

We are currently facing issues when setting up Lakebase autoscaling in a Databricks environment without a public IP, where all traffic is routed privately. We followed the Databricks documentation and configured private endpoints for service direct.

Our Databricks compute can successfully connect to Lakebase using a connection string, and the same applies from machines on our office network. So overall, connectivity is working. However, the problem appears specifically in the Lakebase UI.

When opening the tables view or using the SQL editor in the Lakebase view within the Databricks workspace, the traffic seems to be routed through a non-private endpoint.

What is working:

Accessing Lakebase from notebooks on shared clusters
Accessing Lakebase from serverless notebooks
Accessing Lakebase from our office network
UI features such as branching, creating credentials, and spinning up new Lakebase projects

What is not working:

Tables view and SQL editor in the Lakebase UI

From browser inspection, we see a 403 error on a POST request to:
https://api.database.westeurope.azuredatabricks.net/sql

I have attached:

The error message from the Databricks workspace (tables view)
Network requests from Chrome DevTools showing the failing call

Any ideas what could be missing or misconfigured?

Error on Lakebase Postgres Autoscaling Table Tab

ctgchris — Wed, 08 Apr 2026 22:41:51 GMT

I can't view my lakebase postgres autoscaling project's table tab.
The error id is: 3760588e08b843c5a4aebac770f8e967

"Unknown Error" when running simple select query in Lakebase SQL Editor

Sbolo — Tue, 07 Apr 2026 10:50:59 GMT

Hello! I am experiencing a confusing issue with LakeBase. I created a new project and I tried to run the example query provided in the SQL Editor to create a first table. Unfortunately, the query failed with "Unknown Error" and not further explanation. I then tried to simplify the query further to `SELECT 1`, obtaining the same result (see screenshot). My user has `Can Manage` permissions. I also noticed in the SQL editor that it says "Ready to Connect" in the bottom left corner, but the database claims to be active. I tried waiting around 30 minutes before different attempts at querying the database, and nothing changed.

Potentially related, I can see that, when I try to visualize the tables from the UI, I get the following error:

Server error (HTTP status 403): {"X-Databricks-Reason-Phrase":"Public access is not allowed for workspace: <id here>"}

Please let me know if you can pinpoint what can be going wrong here!

Controlling Agent access to Tools and Tool access to Data Operations

smithsonian — Wed, 01 Apr 2026 13:15:40 GMT

I am building an agentic workflow. This is a multi agent workflow - Plan, Reason, Act and Synthesize. Each agent has its own access to tools to take ACTIONS on data. Some of these agents are READ only, some can WRITE/UPDATE

All data is resident within Unity Catalog when the initial access can be granted.

When the workflow executes and agents come up with dynamic plans I want to control the type of ACTIONS take on the data based on policy contraints. These are runtime Actions I want to control and monitor. Some agents have read only access, some can write (to specific datasets), some can move data (under some conditions).

Is there a best practices approach to control agentic worflows at runtime?

Lakebase -- Enable RLS in synced Table

DaPo — Thu, 04 Sep 2025 13:24:24 GMT

Dear all,

I am currently testing Lakebase for integration in our overall system. In particular I need to enable RLS on a Lakebase table, which is synced from a "Delta Streaming Table" in UC. Setting up the data sync was no trouble, in UC I am the owner of the table. Enabling RLS is somewhat more troublesome. Here is the steps, I have tried (without success):

Create a Database role tied to my databricks User, which has all privileges including databricks_superuser (https://docs.databricks.com/aws/en/oltp/pg-roles)
Open the SQL-Editor, select the Lakebase instance as compute.
Run ALTER TABLE <my-table> ENABLE ROW LEVEL SECURITY; (https://www.postgresql.org/docs/17/ddl-rowsecurity.html)
Get an error: "You must be Owner of the table" (Remember, in UC I am the owner).

I assume, that I am missing something here, how the interaction between databricks user and postgres role happens. So what am I missing?

Also: Are there any possibilities or plans to utilize UC Row-Filters with Lakebase (https://docs.databricks.com/aws/en/data-governance/unity-catalog/filters-and-masks/)? Would be nice to have an unified integrated RLS solution for Lakebase tables and Delta tables.

Greetings, Daniel

Lakebase security

eoferreira — Fri, 15 Aug 2025 17:12:55 GMT

Hi team,

We are using Databricks Enterprise and noticed that our Lakebase instances are exposed to the public internet. They can be reached through the JDBC endpoint with only basic username and password authentication. Is there a way to restrict access to these endpoints? I reviewed the front end Private Link documentation, but it seems that this would cause friction when using the web interface, and I am not sure whether it prevents requests from unwanted IPs from reaching our Lakebase endpoints. If it is possible to configure additional network layer security, we would appreciate any guidance.

Thank you

Lakebase Scale-to-Zero Behavior: Automatic or Application-Controlled?

mrp — Wed, 30 Jul 2025 06:39:29 GMT

Hi all,

Lakebase is currently advertised as a database system that can scale down to zero:
https://www.databricks.com/blog/what-is-a-lakebase

Does anyone know if this scale-to-zero behavior is handled automatically by Databricks when the database is idle, or does it require explicit handling from the application layer?

Lakebase auto start/stop

jericksoncea — Mon, 14 Jul 2025 07:04:31 GMT

It doesn’t appear that the Lakebase OLTP instances function like Sql warehouses in the following ways:
• automatically starting when initiating a connection
• automatically stopping after no usage in x minutes

I am wondering if others have use cases for auto start-stop, and if it on the product roadmap…

Lakebase use cases

Sharanya13 — Wed, 09 Jul 2025 00:21:29 GMT

1. What are the use cases for Lakebase? When should I use the Lakebase Postgres over delta tables?

2. What are the differences between open-source Postgres and Lakebase?

3. Should I utilize Lakebase for all OLTP requirements?