topic Re: Databricks keeps notebooks in its cloud account free of cost? in Get Started Discussions

Databricks keeps notebooks in its cloud account free of cost?

Dhruv-22 — Thu, 25 Jul 2024 09:35:44 GMT

I was reading the architecture of Databricks from the following video. And I came through this

If notebooks are stored in control plane on the databricks cloud account, then does that mean databricks allocates storage for the client free of cost? I just want to confirm that.

Also, is any other thing allocated for the client as well? I know all the compute and storage is billed from the client storage account and there is a workspace storage bucket allocated from the customer account as well. I'm just curious to know if anything else is there in the architecture.

Re: Databricks keeps notebooks in its cloud account free of cost?

mhiltner — Fri, 26 Jul 2024 12:28:30 GMT

Notebooks are stored in the client's cloud storage. The diagram might have confused you, but it just goes to show that you access the notebooks via control plane, not that they're stored there.

Azure Databricks automatically creates the workspace storage account. Azure Databricks uses this storage area for workspace system data and your workspace's DBFS root. Notebook results are stored in workspace system data storage, which is not accessible by users.

Re: Databricks keeps notebooks in its cloud account free of cost?

Dhruv-22 — Thu, 22 Aug 2024 17:19:14 GMT

Hey, the comment you posted talks about notebook results instead of notebooks. Also, the page where the comment is taken (link) says that if the result is small, it is stored in the control plane along with commands. Could this mean that small notebooks are also stored on the control plane? I think the answer about storage location is not either one of control or compute plane. Instead, it is both of them, depending on the size.