When the billing time starts for the cluster?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-11-2021 01:18 AM
Hi All,
I'm just wondering when exactly the billing time starts for the DataBricks cluster? Is starting time included? If cluster creation time takes 3 minutes and query execution only 2, will I pay for 2 or 5?
Thanks in advance!
MC
- Labels:
-
Cluster management
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-11-2021 02:23 AM
Officialy it is for compute time so DBU will be per minute since you start cluster (so 5 in your example). Additionally you will pay for at least 5 minute for instance in your cloud provider (from my experience usually it is a bit more as terminating and cleaning resources takes time). In databricks roadmap there is serverless offering so in future it will be a bit less complicated and more cost effective.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-13-2021 12:15 AM
Correct. The moment your cloud providers starts provisioning and stops provisioning is the time interval you are billed.
So for short jobs that is a lot of overhead.
What I do is to use a pool, in which I try to schedule as many jobs as possible, like that you have a lot less startup time.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-13-2021 10:59 AM
Billing for databricks DBUs starts when Spark Context becomes available. Billing for the cloud provider starts when the request for compute is received and the VMs are starting up.
Stragetic Data and AI Advisor

