topic Re: How to Retrieve DBU Count per Compute Type for Accurate Cost Calculation? in Data Engineering

How to Retrieve DBU Count per Compute Type for Accurate Cost Calculation?

saicharandeepb — Tue, 21 Oct 2025 09:15:33 GMT

Hello Everyone,

We are currently working on a cost analysis initiative to gain deeper insights into our Databricks usage. As part of this effort, we are trying to calculate the hourly cost of each Databricks compute instance by utilizing the Azure Retail Prices API and storing this data in a table for further analysis.

During this process, we encountered a challenge in accurately calculating the cost per compute type. After consulting with Azure Support, we understood that the total cost displayed on Azure Databricks Pricing Page is composed of two components: the VM (infrastructure) cost and the DBU (Databricks Unit) cost.

While we are able to determine the VM cost using the Azure API, we are unable to map the correct number of DBUs per compute type to complete the total cost calculation. We have not found a reliable way to determine the DBU count associated with each VM/compute type.

My questions to the community:

Is there a documented way to retrieve the DBU count per VM or compute type?
Is this information available through any Databricks APIs or system-level tables that we can query?
Has anyone built a similar cost model and can share tips or best practices?

Any guidance or pointers would be really appreciated!

Thanks in advance,

Charan.

Re: How to Retrieve DBU Count per Compute Type for Accurate Cost Calculation?

BS_THE_ANALYST — Tue, 21 Oct 2025 10:55:27 GMT

@saicharandeepb have you looked at the system billing tables in Databricks yet? https://learn.microsoft.com/en-us/azure/databricks/admin/system-tables/billing

There seems to be a field that can display the unit usage in DBU.

Same in this table aswell:
https://learn.microsoft.com/en-us/azure/databricks/admin/system-tables/pricing

All the best,
BS

Re: How to Retrieve DBU Count per Compute Type for Accurate Cost Calculation?

Chiran-Gajula — Tue, 21 Oct 2025 13:32:07 GMT

The majority of the relevant information can be found in the system.billing.usage and system.compute.clusters tables.
To view DBUs by instance type or compute, you can run the following query and explore more of the fields in these tables

SELECT
c.worker_node_type AS vm_instance,
SUM(u.usage_quantity) AS total_dbus
FROM
system.billing.usage u
JOIN
system.compute.clusters c
ON
u.usage_metadata.cluster_id = c.cluster_id
GROUP BY
c.worker_node_type
ORDER BY
total_dbus DESC;

Re: How to Retrieve DBU Count per Compute Type for Accurate Cost Calculation?

nayan_wylde — Tue, 21 Oct 2025 13:49:27 GMT

1. Is there a documented way to retrieve the DBU count per VM or compute type?

Yes, but it's not directly exposed via a single API or table. The DBU consumption rate depends on:

Compute type (Jobs Compute, All-Purpose Compute, SQL Compute, etc.)
VM instance type (e.g., Standard_D16s_v3)
Databricks pricing tier (Standard, Premium, Enterprise)
Cloud provider (Azure, AWS, GCP)

Databricks provides DBU calculators and pricing matrices per cloud provider, which list DBU rates per instance type and workload type. For Azure, you can refer to the https://www.databricks.com/product/pricing/product-pricing/instance-types

2. Is this information available through any Databricks APIs or system-level tables?

Yes. You can use Databricks system tables to analyze DBU consumption:

Key Tables:

system.billing.usage: Tracks DBU usage per workload, SKU, and time window. [learn.microsoft.com]

system.compute.clusters: Contains cluster metadata including node types and configurations.

system.compute.node_types: Maps node types to hardware specs (useful for correlating with DBU rates).

SELECT usage_date, sku_name, usage_quantity AS dbus_consumed, usage_metadata.cluster_id, usage_metadata.job_id FROM system.billing.usage WHERE usage_unit = 'DBU' ORDER BY usage_date DESC;

3. Has anyone built a similar cost model and can share tips or best practices?
Yes, Some of the best practices that I follow are:

Used job clusters + spot instances to reduce DBU and VM costs by up to 50%.
Tagged clusters with environment and workload type for granular cost attribution.
Use cluster policies to restrict high-cost instance types and enforce auto-termination.

Re: How to Retrieve DBU Count per Compute Type for Accurate Cost Calculation?

saicharandeepb — Thu, 23 Oct 2025 06:17:06 GMT

Hi everyone, just to clarify my question — I’m looking for the DBU count per compute type (per instance type), not the total DBU consumption per workload.

In other words, I want to know the fixed DBU rate assigned to each compute SKU (for example, DS3 v2 = 0.75 DBU/hour, DS4 v2 = 1.5 DBU/hour, DS5 v2 = 3.0 DBU/hour, etc.) so that I can accurately estimate costs for different cluster configurations.

I’m not referring to usage or billing metrics that show total DBUs consumed by workloads over time — I just need the reference values that define how many DBUs are billed per hour for each compute type.

Thanks in advance for any guidance.

Re: How to Retrieve DBU Count per Compute Type for Accurate Cost Calculation?

yogeshsingh — Thu, 20 Nov 2025 18:10:02 GMT

Databricks doesn’t expose a “DBU/hour per node type” lookup in system tables, but you can derive it empirically by dividing total DBUs recorded in the billable usage system table by total node runtime from the compute node timeline for each instance type. This gives you the effective DBU/hour per node type for your environment and SKU.

WITH node_minutes AS ( SELECT node_type, COUNT(*) AS minutes -- node_timeline is per-minute records FROM system.compute.node_timeline GROUP BY node_type ), dbu_by_node AS ( SELECT u.usage_metadata.node_type AS node_type, u.sku_name, SUM(u.usage_quantity) AS dbus FROM system.billing.usage AS u WHERE u.usage_unit = 'DBU' AND u.usage_metadata.node_type IS NOT NULL GROUP BY ALL ) SELECT d.node_type, d.sku_name, d.dbus, m.minutes, d.dbus / (m.minutes / 60.0) AS dbu_per_node_hour FROM dbu_by_node d JOIN node_minutes m USING (node_type) ORDER BY dbu_per_node_hour DESC;

Re: How to Retrieve DBU Count per Compute Type for Accurate Cost Calculation?

Coffee77 — Fri, 21 Nov 2025 08:51:57 GMT

You can also take a look at this built-in cost control dashboard explained in the below video or official databricks documentation at https://docs.databricks.com/aws/en/admin/usage/ . Concerning the dashboard, relevant subject for me was you can inspect all the underlying system tables by accessing to que internal data model and queries.