cancel
Showing results for 
Search instead for 
Did you mean: 
Lakebase Discussions
Ask questions, share challenges, and connect with others working on Lakebase. From troubleshooting to best practices, this is where conversations happen.
cancel
Showing results for 
Search instead for 
Did you mean: 

Lakebase Data API – First Request Times Out After 8 Seconds When Compute Is Offline

KeatOoi
New Contributor II

Hi Everyone,

I’m using the Databricks Lakebase Data API and have noticed that the first API call consistently times out after approximately 8 seconds when the Lakebase compute is offline.

The first request triggers the compute to start, and the compute starts successfully, but the API request is cancelled after ~8 seconds. Subsequent requests work normally.

The error is:

{
  "code": "57014",
  "message": "canceling statement due to statement timeout",
  "details": null,
  "hint": null
}

I checked the authenticator role and noticed that the statement_timeout is configured to 8 seconds:

SELECT rolname, rolconfig
FROM pg_roles
WHERE rolname = 'authenticator';

Result:

authenticator | {statement_timeout=8s}

I suspect this statement_timeout may be causing the first request to be cancelled while the compute is starting.

I have permission to alter role, and I tried increasing it to 60 seconds, but received:

ERROR: permission denied to alter role (SQLSTATE 42501)

Has anyone experienced a similar issue? Is there a way to increase or configure the statement_timeout for the Lakebase Data API authenticator role?

Thank you.

4 REPLIES 4

Satyasai
New Contributor II

Try out thes statements

-- Try setting it at the database level
ALTER DATABASE your_database_name SET statement_timeout = '30s';

-- Or set it on the app/claims role that the authenticator assumes
ALTER ROLE your_api_role SET statement_timeout = '30s';

KeatOoi
New Contributor II

I have executed the statements you provided. Still having the same issue, the first api call will time out after 8 seconds.

AbhilashNagilla
Databricks Employee
Databricks Employee

The Data API connects as the authenticator role, which then assumes the requesting user's identity. PostgreSQL loads a role's configuration defaults only at login; SET ROLE and SET SESSION AUTHORIZATION do not load them, and a role-level setting outranks a database-level one (ALTER ROLE). So your ALTER DATABASE change is overridden by the statement_timeout loaded for authenticator, a setting on the assumed user role is never loaded, and authenticator itself cannot be altered.

PostgreSQL counts statement_timeout from when the query reaches the server, so SQLSTATE 57014 confirms a cancellation but does not prove that reactivation consumed the eight seconds.

No Data API setting changes this timeout, so I would:

  1. Enable Server-Timing headers, reproduce the failure, and keep whatever headers appear on the failed response.

  2. Not retry on 57014 alone, since it also covers ordinary cancellation. Retry once only when the request follows a known suspension and is safe to replay, and make writes idempotent first. The scale-to-zero guidance recommends retry logic during reactivation.

  3. For latency-sensitive calls, warm compute with an earlier lightweight read, lengthen the inactivity period, or turn off scale to zero. A longer period reduces how often compute suspends but cannot keep it always active (startup latency).

  4. If it keeps recurring, have an authorized contact open a support case with the project, branch, endpoint, timestamps, the SQLSTATE and message, pg_roles.rolconfig, and any Server-Timing headers.

Below are the server-timing headers metrics that i collected:

MetricSuccessful requestFailed request
JWT0.2 ms25.9 ms
Parse0.0 ms0.1 ms
Plan0.0 ms0.1 ms
Transaction3,844.8 ms8,004.6 ms
Response0.3 ms0.0 ms
ResultSUCCESS57014 TIMEOUT
Authenticator timeout8 seconds8 seconds

So far, what I can confirm is that if a query executed via the Lakebase Data API takes more than 8 seconds, the query will be cancelled.

Therefore, turning off scale-to-zero is not going to solve the issue if the query itself takes longer than 8 seconds.

 

{
    "code": "57014",
    "message": "canceling statement due to statement timeout",
    "details": null,
    "hint": null
}