cancel
Showing results for 
Search instead for 
Did you mean: 
Warehousing & Analytics
Engage in discussions on data warehousing, analytics, and BI solutions within the Databricks Community. Share insights, tips, and best practices for leveraging data for informed decision-making.
cancel
Showing results for 
Search instead for 
Did you mean: 

What Is – a system table? (Your Databricks bank statement)

Valeria_Koz_DBX
Databricks Employee
Databricks Employee

Clients ask me the same three questions all the time:

  • What are system tables?
  • What are they about?
  • And what can I actually do with them?

I recently came across a Databricks blog that answers the last one brilliantly: Top 5 System Table Queries for Understanding Your Databricks Costs. So here's the simple version first. Then go and read it.

The short answer

Your Databricks invoice is the total on your card bill. System tables are the statement behind it: every line, every date, every shop.

They're read-only tables that Databricks fills in for you, in a catalog called system. You query them like any other table, with SQL, a dashboard or Genie.

What's in them

They cover more than cost. Think of them as a building's logbooks:

  • Billing: what you used, and what it costs
  • Access: who opened which door (the audit log)
  • Compute and jobs: which machines ran, what ran on them, and whether it worked

For cost, two tables do the heavy lifting:

  • system.billing.usage is the statement: what was used, how much, and when.
  • system.billing.list_prices is the price tag. Usage is counted in DBUs, not pounds, so you need both.

What you can do with them

The blog gives ready-made SQL for five questions:

  1. Where's the money going? Spend by product, day by day.
  2. Which warehouse is the big spender? And is it growing?
  3. Which team spent it? Like name labels in a shared fridge, this only works if people label things.
  4. Was yesterday weird? Like a smart meter telling you the heating was left on.
  5. Where will we land this quarter? A forecast built on your own history.

I won't repeat the SQL. That's what the blog is for.

Four gotchas

  • Usage isn't money. Multiplying units by the price list gives you the list price, which may not be your negotiated rate.
  • It only covers Databricks. On classic compute, the bill for the cloud provider's machines and storage sits somewhere else.
  • No label, no owner. Untagged spend shows up as "untagged", and that number is the size of your gap.
  • Free to read, not free to run. The tables cost nothing, but the warehouse that queries them does. You may also need an admin to give you access.

Why it matters

One finance team I worked with had all its costs in one unmarked pile. Every budget meeting started with an argument. Once they could see the statement, the meeting started with "here's the line, who owns it?"

Now go and read the blog: Top 5 System Table Queries for Understanding Your Databricks Costs

What's the first question you'd ask your Databricks statement?

2 REPLIES 2

DoTA
Honored Contributor

Nice explainer. To answer your closing question: the first thing I'd ask is what changed week over week by SKU. A cost jump is nearly always one product or one SKU, rarely the whole total moving evenly, so that view finds the culprit fastest.

One gotcha when you join system.billing.usage to system.billing.list_prices: join on the SKU and also on the price window (usage_end_time between price_start_time and price_end_time, where a null end means the current price). If you only join on sku_name you get one row per past price and the totals quietly double up.

Also worth knowing that most system tables only keep about a year of data. If you want longer trends or an audit trail, copy them into your own table on a schedule. Retention differs a little per table, so check the docs for the one you care about.

Valeria_Koz_DBX
Databricks Employee
Databricks Employee

@DoTA , love your comments. All very valid points. Just adding a couple of things: 

1. On the join, I'd write the price window as usage_end_time >= price_start_time AND (price_end_time IS NULL OR usage_end_time < price_end_time) instead of BETWEEN. That's how the docs examples do it, and it stops a row landing on two prices if it ends exactly when a price changes. Small thing, but it helps. And it's list price, so it won't match your invoice if you've got a discount.

2. On retention, most tables keep about a year (365 days), but a few are shorter:

  •  Node timeline and model serving usage keep 90 days
  • MLflow and predictive optimization 180
  • inbound network events only 30
  • and Lakebase observability just 7.
  • Pricing, workspaces and node types are kept forever.
  • There's also a Beta that bumps retention to 395 days if you want year-over-year.

One more tip if you're copying billing.usage into your own table: records can get corrected later (you'll see RETRACTION and RESTATEMENT in record_type). So you should reload the last few days on each run rather than only appending, otherwise your copy slowly drifts from the real numbers.