Databricks Serverless Costs

[Sam500]

Our power BI reports consume real-time data , and for that the only option remains is Databricks serverless,but serverrless is expensive option, how to control the costs for serverless , and any other alternatives. Thank you.

[balajij8]

You can follow below

Serverless Optimization

• Implement Delta Lake optimizations (liquid clustering, Z-ordering) on the tables to reduce scan costs and improve query performance
• Use query result caching - Power BI can reuse cached results for identical queries
• Create materialized views for reporting instead of querying raw data repeatedly. Use partition pruning and predicate pushdown

Refresh Approach

• Validate if real-time is required or if scheduled incremental refreshes (every 30 minutes) meet needs
• Use Power BI's incremental refresh feature to only refresh new/changed data & not full datasets

Alternatives

Provisioned SQL Warehouses

• Classic SQL Warehouses with auto-stop can be more cost-effective for predictable workloads
• For intermittent usage, configure auto stop (10 minutes)
• For continuous usage, configure a right-sized always on warehouse as it generally costs less than serverless

Hybrid

• Use Serverless for ad-hoc queries and exploration
• Use provisioned warehouses for scheduled Power BI refreshes with predictable compute needs

Databricks Streaming & Direct Lake

You can evaluate below if using Azure Databricks

• You can expose curated Delta from Azure Databricks to Fabric One Lake and leverage Direct Lake mode for Power BI reporting. Power BI queries the data directly from the storage for report rendering. Fabric costs will be added to the bill with additional considerations.

Power BI Mode

• Consider Import mode with scheduled refresh instead of Direct Query if real time isn't required
• Consider Composite models - Import aggregated data, Direct Query for details only when needed if possible

[Ashwin_DSA · Accepted solution]

Hi @Sam500,

Before concluding that serverless is expensive, I would first want to understand what that assessment is based on. Have you done a like-for-like benchmarking exercise against classic or pro compute to arrive at that conclusion?

In many cases, people compare only the visible unit cost and stop there. But if you look at the total cost of ownership, serverless is generally lower. Databricks explicitly positions SQL warehouses as the most cost-efficient engine for interactive SQL workloads, and recommends serverless SQL warehouses for most workloads because they start in seconds, scale elastically, and scale down faster when demand drops.

A big reason for this is operational behaviour. Non-serverless warehouses take minutes to start, so in practice, many teams leave them running longer than necessary. Serverless, on the other hand, provides instant availability with automatic scale-up and early scale-down, which often leads to lower overall costs despite the perception that it is the more expensive option.

It is also important to correctly evaluate the full economics. For serverless services, the DBU price already includes the underlying virtual machine cost, so the comparison should be made at the end-to-end workload level, not only on sticker price.

I have personally done this comparison and benchmarking for a number of customers, and the conclusion has consistently been the same.... when measured properly, serverless is usually cheaper than classic compute from a total cost of ownership perspective.

That said, if costs are still higher than expected, the focus should usually be on workload optimisation rather than moving away from serverless. Right-size the warehouse, use aggressive auto-stop, limit max clusters, reduce unnecessary DirectQuery traffic from Power BI, and add proper tagging and monitoring so you can see exactly which reports, users, or models are driving spend.

If this answer resolves your question, could you mark it as “Accept as Solution”? That helps other users quickly find the correct fix.

[kim533]

If your Power BI reports require near real-time data, Serverless SQL can be convenient but expensive at scale. To reduce costs, optimize queries, use aggregation tables, limit data scanned, enable caching, and avoid overly frequent Power BI refreshes. You can also create materialized views or Delta tables with pre-aggregated data so Power BI reads less data. As an alternative, consider using a small all-purpose or SQL warehouse cluster with auto-scaling and auto-stop enabled, which may be more cost-effective for predictable workloads. The best option depends on your query volume, concurrency, and latency requirements, so monitoring Databricks cost reports is essential to identify the biggest cost drivers.

[Yogasathyandrun · Accepted solution]

Serverless is often the preferred option for Power BI DirectQuery workloads because it starts in seconds and scales automatically. However, it’s not always the only option, and there are several ways to reduce costs.

A few high-impact optimizations:

• Set an aggressive auto-stop time (e.g., 5–10 minutes).
• Right-size the warehouse and avoid over-provisioning.
• Optimize queries and reduce unnecessary scans.
• Use aggregated/gold tables instead of querying large raw datasets.
• Run OPTIMIZE and clustering on frequently queried tables.
• Monitor usage through system.billing.usage and set budget alerts.

If your “real-time” requirement is actually a few minutes rather than seconds, consider:

• Power BI Import Mode with incremental refresh.
• Hybrid/Composite models (historical data imported, recent data via DirectQuery).
• Power BI Aggregations.

In many cases, moving part of the workload to Import/Hybrid mode significantly reduces warehouse costs while still providing near real-time reporting.

Could you share:

1. Your freshness requirement (seconds, minutes, hourly)?
2. Approximate concurrent users?
3. Whether you’re using DirectQuery, Import, or a Composite model?

That would help narrow down the most cost-effective approach.