<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>topic Re: SQL Query billing in Warehousing &amp; Analytics</title>
    <link>https://community.databricks.com/t5/warehousing-analytics/sql-query-billing/m-p/165689#M2695</link>
    <description>&lt;P&gt;Hi&lt;STRONG&gt;&lt;a href="https://community.databricks.com/t5/user/viewprofilepage/user-id/229288"&gt;@anmolhhns&lt;/a&gt;,&lt;/STRONG&gt;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;&lt;STRONG&gt;There is no direct per-query cost field in Databricks system tables today.&lt;/STRONG&gt; Here's why, and how to estimate it.&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;&lt;STRONG&gt;The gap:&lt;/STRONG&gt;&lt;/SPAN&gt;&lt;/P&gt;&lt;UL&gt;&lt;LI&gt;&lt;SPAN&gt;system.billing.usage&lt;/SPAN&gt; — records DBU consumption at the &lt;STRONG&gt;warehouse level per hour&lt;/STRONG&gt; (keyed by &lt;SPAN&gt;usage_metadata.warehouse_id&lt;/SPAN&gt; + hourly time window). No &lt;SPAN&gt;statement_id&lt;/SPAN&gt; field.&lt;/LI&gt;&lt;LI&gt;&lt;SPAN&gt;system.query.history&lt;/SPAN&gt; — has detailed per-query metrics (&lt;SPAN&gt;statement_id&lt;/SPAN&gt;, durations, bytes read, etc.) but &lt;STRONG&gt;no cost/DBU column&lt;/STRONG&gt;.&lt;/LI&gt;&lt;/UL&gt;&lt;P&gt;&lt;SPAN&gt;There's no direct join key between billing and an individual query.&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;&lt;STRONG&gt;How to estimate per-query cost (proportional allocation):&lt;/STRONG&gt;&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;The best proxy for "compute work done" by a single query is total_task_duration_ms — defined as &lt;I&gt;"the sum of all task durations across all cores of all nodes"&lt;/I&gt;. You can allocate the hourly warehouse DBU cost proportionally to each query's share of total task time in that hour:&lt;/SPAN&gt;&lt;/P&gt;&lt;LI-CODE lang="markup"&gt;WITH hourly_cost AS (
  SELECT
    usage_metadata.warehouse_id AS warehouse_id,
    DATE_TRUNC('hour', usage_start_time) AS usage_hour,
    SUM(usage_quantity) AS total_dbus,
    SUM(usage_quantity * lp.pricing.default) AS total_cost_usd
  FROM system.billing.usage u
  LEFT JOIN system.billing.list_prices lp
    ON u.sku_name = lp.sku_name AND lp.price_end_time IS NULL
  WHERE usage_metadata.warehouse_id IS NOT NULL
    AND usage_date &amp;gt;= CURRENT_DATE - INTERVAL 7 DAY
  GROUP BY 1, 2
),



hourly_query_work AS (
  SELECT
    compute.warehouse_id AS warehouse_id,
    DATE_TRUNC('hour', start_time) AS query_hour,
    statement_id,
    executed_by,
    statement_text,
    total_task_duration_ms,
    total_duration_ms,
    SUM(total_task_duration_ms) OVER (
      PARTITION BY compute.warehouse_id, DATE_TRUNC('hour', start_time)
    ) AS total_work_in_hour
  FROM system.query.history
  WHERE start_time &amp;gt;= CURRENT_DATE - INTERVAL 7 DAY
    AND execution_status = 'FINISHED'
    AND total_task_duration_ms &amp;gt; 0
)



SELECT
  q.statement_id,
  q.executed_by,
  q.query_hour,
  q.total_duration_ms,
  q.total_task_duration_ms,
  -- Proportional share of the hour's DBU cost
  ROUND(
    c.total_dbus * (q.total_task_duration_ms / q.total_work_in_hour), 4
  ) AS estimated_dbus,
  ROUND(
    c.total_cost_usd * (q.total_task_duration_ms / q.total_work_in_hour), 4
  ) AS estimated_cost_usd,
  LEFT(q.statement_text, 100) AS query_preview
FROM hourly_query_work q
JOIN hourly_cost c
  ON q.warehouse_id = c.warehouse_id
  AND q.query_hour = c.usage_hour
ORDER BY estimated_cost_usd DESC
LIMIT 50&lt;/LI-CODE&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;STRONG&gt;Important caveats:&lt;/STRONG&gt;&lt;/P&gt;&lt;OL&gt;&lt;LI&gt;&lt;STRONG&gt;This is an estimate, not exact billing.&lt;/STRONG&gt;&amp;nbsp;Warehouse DBUs include idle time, cluster startup, and shared overhead that can't be attributed to any single query.&lt;/LI&gt;&lt;LI&gt;&lt;STRONG&gt;Pro/Classic warehouses&lt;/STRONG&gt;&amp;nbsp;bill based on cluster uptime (not per-query), so a lightweight query running alone on an idle warehouse "absorbs" the full hourly cost. Serverless SQL is closer to per-query billing but still doesn't expose it at statement granularity.&lt;/LI&gt;&lt;LI&gt;&lt;STRONG&gt;total_task_duration_ms&lt;/STRONG&gt;&amp;nbsp;is the best available proxy because it reflects actual CPU-seconds consumed across all cores, unlike wall-clock duration which includes queueing.&lt;/LI&gt;&lt;LI&gt;&lt;STRONG&gt;Retention:&lt;/STRONG&gt;&amp;nbsp;system.query.history&amp;nbsp;retains 30 days;&amp;nbsp;system.billing.usage&amp;nbsp;retains 365 days.&lt;/LI&gt;&lt;/OL&gt;&lt;P&gt;&lt;STRONG&gt;Feature request:&lt;/STRONG&gt;&amp;nbsp;If exact per-query cost attribution is critical for your use case (chargeback, showback), I'd recommend filing a request on the&amp;nbsp;&lt;A title="https://ideas.databricks.com/" href="https://ideas.databricks.com/" rel="noreferrer noopener" target="_blank"&gt;Databricks Ideas Portal&lt;/A&gt;&amp;nbsp;— it's a frequently requested feature.&lt;/P&gt;&lt;P&gt;&lt;STRONG&gt;If my answer was helpful, please consider marking it as accepted solution!&lt;/STRONG&gt;&lt;/P&gt;</description>
    <pubDate>Fri, 14 Aug 2026 18:09:41 GMT</pubDate>
    <dc:creator>GabFernandes</dc:creator>
    <dc:date>2026-08-14T18:09:41Z</dc:date>
    <item>
      <title>SQL Query billing</title>
      <link>https://community.databricks.com/t5/warehousing-analytics/sql-query-billing/m-p/165679#M2694</link>
      <description>&lt;P&gt;Is there any way to find a cost of one single query? i can't find any table in system catalog to connect a query with usage to find the exact cost of one query run in sql warehouse?&lt;/P&gt;</description>
      <pubDate>Fri, 14 Aug 2026 15:51:29 GMT</pubDate>
      <guid>https://community.databricks.com/t5/warehousing-analytics/sql-query-billing/m-p/165679#M2694</guid>
      <dc:creator>anmolhhns</dc:creator>
      <dc:date>2026-08-14T15:51:29Z</dc:date>
    </item>
    <item>
      <title>Re: SQL Query billing</title>
      <link>https://community.databricks.com/t5/warehousing-analytics/sql-query-billing/m-p/165689#M2695</link>
      <description>&lt;P&gt;Hi&lt;STRONG&gt;&lt;a href="https://community.databricks.com/t5/user/viewprofilepage/user-id/229288"&gt;@anmolhhns&lt;/a&gt;,&lt;/STRONG&gt;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;&lt;STRONG&gt;There is no direct per-query cost field in Databricks system tables today.&lt;/STRONG&gt; Here's why, and how to estimate it.&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;&lt;STRONG&gt;The gap:&lt;/STRONG&gt;&lt;/SPAN&gt;&lt;/P&gt;&lt;UL&gt;&lt;LI&gt;&lt;SPAN&gt;system.billing.usage&lt;/SPAN&gt; — records DBU consumption at the &lt;STRONG&gt;warehouse level per hour&lt;/STRONG&gt; (keyed by &lt;SPAN&gt;usage_metadata.warehouse_id&lt;/SPAN&gt; + hourly time window). No &lt;SPAN&gt;statement_id&lt;/SPAN&gt; field.&lt;/LI&gt;&lt;LI&gt;&lt;SPAN&gt;system.query.history&lt;/SPAN&gt; — has detailed per-query metrics (&lt;SPAN&gt;statement_id&lt;/SPAN&gt;, durations, bytes read, etc.) but &lt;STRONG&gt;no cost/DBU column&lt;/STRONG&gt;.&lt;/LI&gt;&lt;/UL&gt;&lt;P&gt;&lt;SPAN&gt;There's no direct join key between billing and an individual query.&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;&lt;STRONG&gt;How to estimate per-query cost (proportional allocation):&lt;/STRONG&gt;&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;The best proxy for "compute work done" by a single query is total_task_duration_ms — defined as &lt;I&gt;"the sum of all task durations across all cores of all nodes"&lt;/I&gt;. You can allocate the hourly warehouse DBU cost proportionally to each query's share of total task time in that hour:&lt;/SPAN&gt;&lt;/P&gt;&lt;LI-CODE lang="markup"&gt;WITH hourly_cost AS (
  SELECT
    usage_metadata.warehouse_id AS warehouse_id,
    DATE_TRUNC('hour', usage_start_time) AS usage_hour,
    SUM(usage_quantity) AS total_dbus,
    SUM(usage_quantity * lp.pricing.default) AS total_cost_usd
  FROM system.billing.usage u
  LEFT JOIN system.billing.list_prices lp
    ON u.sku_name = lp.sku_name AND lp.price_end_time IS NULL
  WHERE usage_metadata.warehouse_id IS NOT NULL
    AND usage_date &amp;gt;= CURRENT_DATE - INTERVAL 7 DAY
  GROUP BY 1, 2
),



hourly_query_work AS (
  SELECT
    compute.warehouse_id AS warehouse_id,
    DATE_TRUNC('hour', start_time) AS query_hour,
    statement_id,
    executed_by,
    statement_text,
    total_task_duration_ms,
    total_duration_ms,
    SUM(total_task_duration_ms) OVER (
      PARTITION BY compute.warehouse_id, DATE_TRUNC('hour', start_time)
    ) AS total_work_in_hour
  FROM system.query.history
  WHERE start_time &amp;gt;= CURRENT_DATE - INTERVAL 7 DAY
    AND execution_status = 'FINISHED'
    AND total_task_duration_ms &amp;gt; 0
)



SELECT
  q.statement_id,
  q.executed_by,
  q.query_hour,
  q.total_duration_ms,
  q.total_task_duration_ms,
  -- Proportional share of the hour's DBU cost
  ROUND(
    c.total_dbus * (q.total_task_duration_ms / q.total_work_in_hour), 4
  ) AS estimated_dbus,
  ROUND(
    c.total_cost_usd * (q.total_task_duration_ms / q.total_work_in_hour), 4
  ) AS estimated_cost_usd,
  LEFT(q.statement_text, 100) AS query_preview
FROM hourly_query_work q
JOIN hourly_cost c
  ON q.warehouse_id = c.warehouse_id
  AND q.query_hour = c.usage_hour
ORDER BY estimated_cost_usd DESC
LIMIT 50&lt;/LI-CODE&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;STRONG&gt;Important caveats:&lt;/STRONG&gt;&lt;/P&gt;&lt;OL&gt;&lt;LI&gt;&lt;STRONG&gt;This is an estimate, not exact billing.&lt;/STRONG&gt;&amp;nbsp;Warehouse DBUs include idle time, cluster startup, and shared overhead that can't be attributed to any single query.&lt;/LI&gt;&lt;LI&gt;&lt;STRONG&gt;Pro/Classic warehouses&lt;/STRONG&gt;&amp;nbsp;bill based on cluster uptime (not per-query), so a lightweight query running alone on an idle warehouse "absorbs" the full hourly cost. Serverless SQL is closer to per-query billing but still doesn't expose it at statement granularity.&lt;/LI&gt;&lt;LI&gt;&lt;STRONG&gt;total_task_duration_ms&lt;/STRONG&gt;&amp;nbsp;is the best available proxy because it reflects actual CPU-seconds consumed across all cores, unlike wall-clock duration which includes queueing.&lt;/LI&gt;&lt;LI&gt;&lt;STRONG&gt;Retention:&lt;/STRONG&gt;&amp;nbsp;system.query.history&amp;nbsp;retains 30 days;&amp;nbsp;system.billing.usage&amp;nbsp;retains 365 days.&lt;/LI&gt;&lt;/OL&gt;&lt;P&gt;&lt;STRONG&gt;Feature request:&lt;/STRONG&gt;&amp;nbsp;If exact per-query cost attribution is critical for your use case (chargeback, showback), I'd recommend filing a request on the&amp;nbsp;&lt;A title="https://ideas.databricks.com/" href="https://ideas.databricks.com/" rel="noreferrer noopener" target="_blank"&gt;Databricks Ideas Portal&lt;/A&gt;&amp;nbsp;— it's a frequently requested feature.&lt;/P&gt;&lt;P&gt;&lt;STRONG&gt;If my answer was helpful, please consider marking it as accepted solution!&lt;/STRONG&gt;&lt;/P&gt;</description>
      <pubDate>Fri, 14 Aug 2026 18:09:41 GMT</pubDate>
      <guid>https://community.databricks.com/t5/warehousing-analytics/sql-query-billing/m-p/165689#M2695</guid>
      <dc:creator>GabFernandes</dc:creator>
      <dc:date>2026-08-14T18:09:41Z</dc:date>
    </item>
    <item>
      <title>Re: SQL Query billing</title>
      <link>https://community.databricks.com/t5/warehousing-analytics/sql-query-billing/m-p/166948#M2706</link>
      <description>&lt;P data-pm-slice="1 1 []"&gt;The published system tables do not expose the exact cost of an individual SQL statement. &lt;CODE&gt;&lt;A href="https://docs.databricks.com/aws/en/admin/system-tables/billing" target="_self"&gt;system.billing.usage&lt;/A&gt;&lt;/CODE&gt; records warehouse-attributed usage for each record's time interval, but has no statement or session ID. &lt;CODE&gt;&lt;A href="https://docs.databricks.com/aws/en/admin/system-tables/query-history" target="_self"&gt;system.query.history&lt;/A&gt;&lt;/CODE&gt; records &lt;CODE&gt;statement_id&lt;/CODE&gt;, &lt;CODE&gt;compute.warehouse_id&lt;/CODE&gt;, and timing metrics, but has no billing-record key, cost, or DBU column; warehouse ID plus overlapping time can correlate the tables, but cannot uniquely join a billing record to a statement.&lt;/P&gt;
&lt;P&gt;For showback, you can define a heuristic that allocates each interval's warehouse cost among overlapping statements, with &lt;CODE&gt;total_task_duration_ms&lt;/CODE&gt; as one possible weighting factor. Databricks documents that metric's meaning and the &lt;A href="https://docs.databricks.com/aws/en/admin/usage/system-tables" target="_blank"&gt;effective-date join from usage to list prices&lt;/A&gt;, but does not prescribe a per-query allocation formula. Concurrency, autoscaling, cache hits, shared idle time, and queries spanning intervals can skew the allocation, and published list cost can differ from invoice cost.&lt;/P&gt;
&lt;P&gt;The &lt;A href="https://docs.databricks.com/aws/en/admin/system-tables/" target="_blank"&gt;system tables reference&lt;/A&gt; lists a 365-day free retention period for both tables.&lt;/P&gt;</description>
      <pubDate>Mon, 31 Aug 2026 16:59:20 GMT</pubDate>
      <guid>https://community.databricks.com/t5/warehousing-analytics/sql-query-billing/m-p/166948#M2706</guid>
      <dc:creator>AbhilashNagilla</dc:creator>
      <dc:date>2026-08-31T16:59:20Z</dc:date>
    </item>
  </channel>
</rss>

