How to Know DBU consumption in azure databricks ?

[Kroy]

In Azure portal - Billing we can get the COST but how to know How much DBU is consumed ?

[Kaniz]

Hi @Kroy, To determine the Databricks Units (DBU) consumption in Azure Databricks, you can follow these steps:

Understanding DBUs:

• DBUs represent a unit of processing capability in Azure Databricks.
• They are billed on a per-second usage basis.
• The actual DBU consumption depends on the size and type of instance running in your Azure Databricks....

DBU Calculation:

• The DBU consumption is influenced by the number of workers utilized in your cluster.
• Each worker is allocated a DBU/h (DBU per hour) of 0.75 DBU/h2.
• Therefore:
  • For 1 worker, the DBU consumption is 1.5 DBU/h.
  • For 2 workers, the DBU consumption is 2.25 DBU/h.
  • And so on, based on the number of workers2.

Auto-Scaling and DBUs:

• If you have autoscaling enabled, the DBU consumption dynamically adjusts based on the number of workers being utilized.
• When autoscaling is disabled, a fixed number of workers are assigned to the cluster, each with its r....

Monitoring DBU Usage:

• To monitor your DBU usage, consider the following:
  • Cluster Summary: In the Databricks UI, check the cluster summary. It displays the DBU range (e.g., 1.5-2.25 DBU/h) based on the workers used2.
  • Usage Data CSV: You can download a usage data CSV file from Databricks and analyze it to understand ...3.

If you have any further questions, feel free to ask! 😊

[Kroy]

Hi @Kaniz  - 

I got hand on system table - which tells how much DBU is consumed

system.billing.usage

but problem is for example - serverless_sql_compute  here of 05th March it says DBU usage is 20.82 but as per rate if I combine 20.82 * 0.91 $ it does not match to what I see in Azure portal, it is much less in azure billing page