Databricks Job cost(AWS)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-05-2024 11:15 PM
Hi Databricks Community,
I am looking for a formula/way to calculate the estimated cost for a job run, for which I have a few questions:
1. Is there any formula to calculate the cost of any job like -> [(EC2 per hr cost) * (total time job ran)]
and when i say total time job ran, do we subtract the 5 min which usually takes for a cluster to start, cause for an hourly job that 5 min would end up being extra 2hrs per day
2. Do we include DBU consumed too when calculating cost ->[(EC2 per hr cost) * (total time job ran)]+[(DBU consumed)*(DBU/hr rate)]
In conclusion i would like to know the formula for calculating a Job run cost(using AWS EC2) and if we need to include the 5 min of cluster starting time in formula
Thanks
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-06-2024 01:12 AM
Here's how it works:
- You pay for DBUs when the cluster is up and running up to the point when cluster is terminated.
- You pay for VM from start to deallocation of VM.
It's quite good explained on Azure pricing documentation. It's about instance pools, but gives a fine overview of how it actually works:
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-06-2024 09:44 AM - edited 03-06-2024 09:44 AM
This looks a little bit confusing to me, I'm looking for a more straight forward answer, more like a simple formula
Thanks though for your reply

