I'm exploring using an instance pool. Can someone clarify for me which job event log tells me the time-spent-waiting-for-an-instance? I've found 2 candidates:
1. The delta between "waitingForCluster" and "started" on the "run events" log, accessible via "View Run Events" button on the Runs page (e.g.: `https://tecton-dev-shared-databricks.cloud.databricks.com/jobs/12345/runs/12345`):
2. The delta between "CREATING" and "INIT_SCRIPTS_STARTED" on the "run events" tab on the Compute page (e.g.: `https://tecton-dev-shared-databricks.cloud.databricks.com/compute/clusters/12345/events?o=12345`:
Which delta should I use to assess time-spent-waiting-for-an-instance?
Hey @mrstevegross
I think you are reading the “View Run Events” log in reverse order.
I’m attaching an example where I compare the View Run Events (Right) with the Run Events from the Compute Page (Left).
As you can see, both logs contain the same information, but in the correct order:
CREATING → INIT_SCRIPTS_STARTED (20:44:54 → 20:46:58) ≈ 2 min
If your setup doesn’t include init scripts, the time should be slightly lower, as the cluster wouldn’t need to execute those steps.
Hope this helps 🙂
Isi
