We've been digging into serverless interactive compute costs in notebooks, because our internal cost monitoring (cross-referencing system.billing.usage with system.query.history) showed a large share of the spend coming from idle time rather than actual query execution. To understand why, we wanted to inspect the actual cluster configuration behind a serverless notebook session — but the notebook UI doesn't expose a cluster ID anywhere for serverless, so there's no obvious way to query it via API or SDK. 

How we found the serverless cluster ID (not exposed in the UI):

The workaround: open your browser's DevTools → Network tab, attach the notebook to Serverless, and run any cell. This triggers a request to the internal notebook/{notebook_id}/command endpoint. In that request's headers, look at x-databricks-attribution-tags — it's a URL-encoded JSON blob, and one of its fields (clusterId) contains the serverless cluster ID generated for that session. With that ID, you can query the cluster programmatically via the SDK:

import os

from databricks.sdk import WorkspaceClient
from databricks.sdk.service import compute

w = WorkspaceClient()

cluster_id = "your_serverless_cluster_id"

cluster = w.clusters.get(cluster_id)

print(cluster.autotermination_minutes)

The actual finding:

Querying the cluster this way, I see that autotermination_minutes defaults to 60 minutes — the same value historically used for classic all-purpose clusters, which take several minutes to spin up (which is why it makes sense not to tear down a classic cluster too quickly).

But serverless compute has the opposite value proposition: near-instant startup, with no provisioning cost for the user to wait through. That should allow for a much more aggressive idle timeout — without the "wait for the cluster to come back up" trade-off that justifies 60 minutes on classic clusters.

In practice, this generates real waste: we cross-referenced system.billing.usage with system.query.history to measure real usage vs. idle time in developer sessions, and in one specific case we found 99.7% of billed time was idle — a user who ran just over 1 minute of actual commands ended up billed for a 6-hour session, because the notebook was left open without ever hitting "Terminate."

When I query interactive serverless sessions for a user directly against the billing table, I can clearly see several sessions lasting exactly 60 minutes.

%sql
-- 'Query generated by genie code'
WITH prices AS (
  SELECT
    sku_name,
    usage_unit,
    price_start_time,
    COALESCE(price_end_time, date_add(current_date, 1)) AS price_end_time_eff,
    pricing.default AS unit_price
  FROM system.billing.list_prices
  WHERE currency_code = 'USD'
)
SELECT
  u.usage_start_time,
  u.usage_end_time,
  ROUND(timestampdiff(SECOND, u.usage_start_time, u.usage_end_time) / 60.0, 2) AS duration_minutes,
  u.sku_name,
  ROUND(u.usage_quantity, 4) AS dbu,
  ROUND(u.usage_quantity * p.unit_price, 4) AS cost_usd,
  u.usage_metadata.notebook_id AS notebook_id,
  u.usage_metadata.notebook_path AS notebook_path
FROM system.billing.usage u
  LEFT JOIN prices p
    ON u.sku_name = p.sku_name
    AND u.usage_unit = p.usage_unit
    AND u.usage_end_time BETWEEN p.price_start_time AND p.price_end_time_eff
WHERE
  u.billing_origin_product = 'INTERACTIVE'
  AND u.identity_metadata.run_as = 'USER_EMAIL'
  AND u.usage_date = '2026-08-10'
  AND u.workspace_id IN ('WORKSPACE_ID_1', 'WORKSPACE_ID_2')
ORDER BY u.usage_start_time

So, why is the default auto-termination for serverless interactive notebook compute 60 minutes?