Resource limits

AnhPT
New Contributor III

Thursday

Hi forks,
I see page https://docs.databricks.com/aws/en/resources/limits
Does Databricks have dashboard concurrent limits and resource limits ? I am wondering whether these limits are calculated over a time window of 1 day, 1 hour, or 1 minute. Please share your insights with me.

0 Kudos

AnhPT
New Contributor III
In response to Satyasai

yesterday

@Satyasai  documents not detail.

0 Kudos

Louis_Frolio
Databricks Employee
Databricks Employee

yesterday

Hello @AnhPT, I took a look at both internal and external documentation and here is what I found. @Satyasai pointed you at the right page for dashboards; the reason it felt thin is that the time-window question doesn't apply to most of what's on it.

The resource limits page mixes three kinds of limits, and only one has a time window.

Static counts cap how many of something can exist: 1,000 SQL warehouses per workspace, 15 pages and 100 datasets per dashboard, 100 widgets per page. No day, hour, or minute involved.

Concurrency limits cap how many things can run at the same instant: 2,000 tasks running simultaneously per workspace, 1,000 queries in a SQL warehouse queue. Also no window. The check happens when you try to start one more.

Rate limits are the only ones with a window, and it's written into the Metric column of the API rate limits table. Most are per second (/jobs/create is 20 per second), a few are per minute (Secrets API is 1,100 per minute), and the lineage API is per hour and per day. There's also one per-hour limit in the resource table: 10,000 jobs created per hour per workspace.

So the straight answer is: read the metric name on the row. If it doesn't name a window, it isn't a rate limit and the question doesn't apply.

On dashboards specifically, the documented limits are object, rendering, and subscription limits, not quotas. Most charts render up to 15,000 rows, tables up to 100,000, subscription attachments are capped at 9 MB combined, and the 100,000-row or 100 MB dataset threshold is a performance switch (small datasets filter in the browser, large ones rerun on the backend), not a hard cap. Neither page publishes a limit on concurrent dashboard viewers or concurrent dashboard queries. Dashboards don't have their own concurrency ceiling. The SQL warehouse behind them does.

  • Classic and pro warehouses have a fixed limit of one cluster per 10 concurrent queries. Beyond that, queries queue (max 1,000 for any warehouse type). Autoscaling adds clusters based on estimated backlog, scales up if a query waits 5 minutes, and scales down after 15 minutes of low load.
  • Serverless uses Intelligent Workload Management to predict each query's cost, admit or queue it, and scale out when queue wait climbs. Same 1,000 queue cap.
  • The number to watch is Peak Queued Queries on the warehouse monitoring tab. Steady above zero means a bigger size or higher max cluster count.

One thing that changes the math: a dashboard view isn't a warehouse query. Published dashboards keep a 24-hour result cache (best effort). With shared data permissions, every viewer reads the same cache, so 200 people opening it at 9am might cost one query per dataset, not 200. With individual permissions, or on the draft, the cache is per user. A schedule warms the cache before people arrive, and dataset materialization precomputes results outright.

Practical version: size max clusters for peak concurrent queries (roughly 10 per cluster on pro or classic), publish with shared data permissions plus a schedule, keep datasets under the small threshold where you can, and watch Peak Queued Queries. If you're seeing a specific error, post the exact message and whether it happens while opening, refreshing, querying, or subscribing to the dashboard, and we can narrow it down. If there are internal throttles on the dashboard service itself, they aren't published, so I can't give you a number for them.

References:

Regards, Louis.

0 Kudos