topic Re: Databricks All-purpose compute Pricing in Administration & Architecture

Databricks All-purpose compute Pricing

karen_c — Fri, 06 Dec 2024 06:46:23 GMT

Hello, I am now struggling how to calculate the cost of my job cluster.

My configuration is as below:

If I have to run the above cluster 18 hours per day, in Standard Tier and East Asia Region, how much will be the pricing of the cluster?

Any help provided is greatly appreciated!! Please provide details if possible!!

Thanks 🙂

Re: Databricks All-purpose compute Pricing

Takuya-Omi — Fri, 06 Dec 2024 09:12:41 GMT

@karen_c

The cost of Azure Databricks is primarily composed of the following three elements:

Azure Virtual Machine (VM) instance cost: Charged based on the type and size of the VM used for data processing.
Databricks Unit (DBU) cost: The software usage fee for Databricks, which varies depending on the workload type and cluster size.

In summary, the total cost can be calculated as:
Total Cost = (VM Instance Cost + DBU Cost) × Usage Hours

Strictly speaking, additional costs such as storage services are also required, but I’ve omitted them for simplicity.

This pricing can be easily calculated using Azure’s official documentation.

https://azure.microsoft.com/en-us/pricing/details/databricks/#pricing

Based on your specific conditions, I’ve done some research. The key section to look at is the "Pay As You Go Total Price."

It states $1.084/hour. However, since you’re using 1 driver node and 2 worker nodes (a total of 3 nodes), the cost becomes $3.252/hour. Running this for 18 hours would result in a total of $58.536.

One important note is that enabling Photon affects the DBU cost, which in turn impacts the overall cost. When calculating, be sure to select "with Photon," as shown in the attached image.

Please double-check my calculations, as there’s a chance I might have made a mistake. I hope this information is helpful to you!

Re: Databricks All-purpose compute Pricing

Takuya-Omi — Fri, 06 Dec 2024 09:17:06 GMT

@karen_c

Let me make a small correction.

It seems that you have checked the option for Spot Instances, which should make the cost slightly lower. Please refer to the far-right column of the attached pricing table for more details.

Additionally, you have set your cluster to autoscale, allowing worker nodes to scale up to a maximum of 4 nodes. My calculation was based on the assumption that the cluster runs with a fixed 2 worker nodes, without scaling. Please keep this difference in mind as well.