When the billing time starts for the cluster?

[Mariusz_Cyp]

Hi All,

I'm just wondering when exactly the billing time starts for the DataBricks cluster? Is starting time included? If cluster creation time takes 3 minutes and query execution only 2, will I pay for 2 or 5?

Thanks in advance!

MC

[Hubert-Dudek]

Officialy it is for compute time so DBU will be per minute since you start cluster (so 5 in your example). Additionally you will pay for at least 5 minute for instance in your cloud provider (from my experience usually it is a bit more as terminating and cleaning resources takes time). In databricks roadmap there is serverless offering so in future it will be a bit less complicated and more cost effective.

[-werners-]

Correct. The moment your cloud providers starts provisioning and stops provisioning is the time interval you are billed.

So for short jobs that is a lot of overhead.

What I do is to use a pool, in which I try to schedule as many jobs as possible, like that you have a lot less startup time.

[franco_patano]

Billing for databricks DBUs starts when Spark Context becomes available. Billing for the cloud provider starts when the request for compute is received and the VMs are starting up.