topic Re: Serverless Compute in Data Engineering

Serverless Compute

neeraj_borana — Thu, 05 Feb 2026 15:06:37 GMT

Is there a way to create multiple serverless compute instances in a Databricks workspace?

I have a use case where multiple all-purpose clusters are allocated to different user groups. I want to replace these all-purpose clusters with serverless compute. Is it possible to allocate or logically separate serverless compute for different user groups?
Additionally, is it possible to limit or control serverless compute usage per user group or workspace?

Re: Serverless Compute

nayan_wylde — Thu, 05 Feb 2026 15:30:30 GMT

Not really. There isn’t an admin-managed concept like “Serverless Cluster A for Team A” and “Serverless Cluster B for Team B” for notebooks/jobs serverless.

Serverless compute is described as Databricks-managed on-demand resources where Databricks “automatically allocates and manages the necessary compute resources” instead of you provisioning/creating clusters.

If it is for cost tracking. Databricks provides Serverless budget policies (public preview) that apply tags to serverless activity for users/groups assigned to those policies. Those tags then appear in billing records, enabling chargeback/showback by team/cost center.
https://docs.databricks.com/aws/en/admin/usage/budget-policies

Re: Serverless Compute

neeraj_borana — Thu, 05 Feb 2026 17:40:00 GMT

I want limit or control serverless compute usage per user group? Is there any way to do that?

Re: Serverless Compute

nayan_wylde — Thu, 05 Feb 2026 17:56:17 GMT

No — Databricks does not currently offer a native way to set hard quotas or throttles on serverless compute per user group (for example, “Team A is limited to X DBUs/hour on serverless notebooks”). Serverless compute is intentionally a shared, autoscaling service managed by Databricks.
What is available today are governance and cost‑control mechanisms, not hard enforcement.

Some recommended best practices around serverless are:
Serverless budget policies for per‑group attribution and monitoring
Budget alerts + reporting (FinOps dashboards, system tables)