nayan_wylde
Esteemed Contributor II

‎10-21-2025 06:49 AM

1. Is there a documented way to retrieve the DBU count per VM or compute type?

Yes, but it's not directly exposed via a single API or table. The DBU consumption rate depends on:

Compute type (Jobs Compute, All-Purpose Compute, SQL Compute, etc.)
VM instance type (e.g., Standard_D16s_v3)
Databricks pricing tier (Standard, Premium, Enterprise)
Cloud provider (Azure, AWS, GCP)

Databricks provides DBU calculators and pricing matrices per cloud provider, which list DBU rates per instance type and workload type. For Azure, you can refer to the https://www.databricks.com/product/pricing/product-pricing/instance-types

 

 

2. Is this information available through any Databricks APIs or system-level tables?
Yes. You can use Databricks system tables to analyze DBU consumption:
Key Tables:
 
system.billing.usage: Tracks DBU usage per workload, SKU, and time window. [learn.microsoft.com]
system.compute.clusters: Contains cluster metadata including node types and configurations.
system.compute.node_types: Maps node types to hardware specs (useful for correlating with DBU rates).
 
SELECT 
  usage_date,
  sku_name,
  usage_quantity AS dbus_consumed,
  usage_metadata.cluster_id,
  usage_metadata.job_id
FROM system.billing.usage
WHERE usage_unit = 'DBU'
ORDER BY usage_date DESC;

3. Has anyone built a similar cost model and can share tips or best practices?
Yes, Some of the best practices that I follow are:

  • Used job clusters + spot instances to reduce DBU and VM costs by up to 50%.
  • Tagged clusters with environment and workload type for granular cost attribution.
  • Use cluster policies to restrict high-cost instance types and enforce auto-termination.

 

0 Kudos