- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-07-2026 08:39 AM
Hi @Malthe,
Thanks for sharing the extra snapshots and observations.
You’re absolutely right that the core‑hour math and the 8 DBUs line up. The issue you’re hitting is that the query profile UI isn’t a fully additive cost‑accounting view, especially for complex DML and serverless.
A few clarifications based on how the system works today:
- The “Aggregated task time” metric is the sum of task time across all cores and all tasks/attempts for the query, and it’s expected to diverge from wall‑clock when there’s parallelism or waiting for resources. That’s the number that matches your core‑hour/DBU math.
- The execution dropdown and per‑operator boxes are aggregated differently:
- Some non‑Photon and high‑level operations (like INSERT / MERGE wrappers) are grouped and share metrics with a parent operator, so the child or parent can show “0 ms / 0 bytes” even though work happened underneath.
- By default, metrics for certain operations are hidden, and only show up when you enable verbose mode in the profile.
- Because of those aggregation choices, the sum of execution times and box times doesn’t necessarily equal the global aggregated task time or wall‑clock, which is why it feels non‑reconcilable
- For a more Spark‑UI‑style breakdown, you can:
- Open the run in the Spark UI from the query profile (kebab menu → Open in Spark UI).
- Use the Query History API together with the system billing tables to correlate query/task metrics with DBUs at whatever granularity you need.
So from a billing perspective, the DBUs you’re seeing are consistent with the aggregated task time. From a transparency perspective, your criticism is fair. The current query profile is optimised for finding bottlenecks, not yet for reconstructing every billed minute. Feedback like this is exactly what we’re using to improve the serverless profiling and cost‑attribution experience.
If this answer resolves your question, could you mark it as “Accept as Solution”? That helps other users quickly find the correct fix.
Ashwin | Delivery Solution Architect @ Databricks
Helping you build and scale the Data Intelligence Platform.
***Opinions are my own***