cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Inquiry Regarding Serverless Compute Operations After Cloud Account Suspension

Harun
Honored Contributor

Hello Everyone,

I am currently benchmarking the new serverless compute feature and have observed an unexpected behavior under specific circumstances. During my benchmarking process, I executed two notebooks: one utilizing serverless compute and the other using all-purpose compute.

The notebook attached to the serverless compute completed its execution in 20 minutes, while the notebook using all-purpose compute ran for over three hours. Unfortunately, I exhausted all my Azure credits during this period.

After my Azure cloud account was disabled, I found that I could no longer create any new all-purpose compute resources, which is expected. However, I am still able to start and use serverless compute, including both notebooks and SQL Data Warehouse serverless resources. I can perform transformations and use commands like df.show() to view dataframe results, but I am unable to use df.display() or write to cloud destinations.

My question is twofold:

  1. Hosting and Operations: Where are these serverless compute resources hosted if my cloud subscription is disabled?
  2. Billing Concerns: How is the billing for these serverless compute usages handled if my account is no longer active?

I would appreciate any insights or clarifications on this.

Attaching snapshots of the serverless computes that are still operational, as well as the failure encountered when trying to create all-purpose compute resources after the cloud account was suspended

Harun_0-1723183858489.png

Harun_1-1723183868878.png

 

1 REPLY 1

mark_ott
Databricks Employee
Databricks Employee

Serverless compute resources in Azure Databricks and Azure SQL can operate independently of your cloud subscription state because they are fully managed, abstracted services that run on infrastructure controlled by Azure rather than your own cloud account’s resources or VNet. This separation leads to the specific behaviors you observed after your Azure account was disabled.​

Hosting and Operations

Serverless compute (for Databricks notebooks or SQL Data Warehouse) is hosted in Azure-managed environments, meaning workloads do not require direct provisioning of underlying VMs, clusters, or network resources in your tenant. When your cloud subscription is disabled, all-purpose compute and other tenant-bound services stop functioning because they require access to resources billed directly from your account. However, serverless compute resources are provisioned and orchestrated entirely by Azure Databricks (or SQL serverless), allowing you to continue executing supported actions like code computations and in-memory dataframe manipulations (e.g., df.show()). Actions involving cloud storage or external endpoints (such as writing results or using df.display(), which often requires enhanced UI integration) may be restricted, since those operations require active connections and credentials through your subscription.​

Billing Concerns

Billing for serverless compute is handled by Azure’s central managed platform. If your Azure subscription is disabled (due to exhausted credits or payment lapse), you lose the ability to create or manage tenant-based, all-purpose compute resources—these require an active billing relationship with Azure. For serverless compute, billing is usage-based and tracks workloads run on Azure-managed VMs. If your account cannot be billed, further usage of serverless compute should be restricted, or only previously paid/free-tier operations may be allowed. In some cases, you may be able to run workloads temporarily until the account is fully locked, but any new charges will be queued until the subscription is reactivated or the outstanding balance is paid.​

If you are continuing to use serverless compute after your account was disabled, you are likely operating in a temporary grace period granted by Azure, during which only platform-managed workloads—not direct cloud resource creation or external writes—can be executed. As soon as the grace period ends or further billing is required, compute will be fully paused.​

Summary Table

Compute Type Hosting Location Functionality After Account Disabled Billing Status
All-purpose compute User’s Azure cloud resources Cannot start or create new resources Billing directly from account; stops when disabled​
Serverless compute Azure-managed infrastructure Can start & run compute; limited external actions Usage metered by Azure; subject to grace period or queued charges​
 
 

Azure-managed serverless compute continues to operate with limited functionality after account disablement because its infrastructure is orchestrated independently of your subscription; billing may persist up to the grace-period limit or until account reactivation, and new resource creation is blocked until billing issues are resolved.