using Azure Databricks vs using Databricks directly

[abin-bcgov]

Hi friends,

A quick question regarding how data, workspace controls works while using "Azure Databricks". I am planning to use Azure Databricks that comes as part of my employer's Azure Subscriptions. I work for a Public sector organization, which is BC Government , Canada. We use services from Azure Portal , that is governed by guardrail mechanism, in terms to geolocation, data residency etc. To be specific , we created all Azure Resources that is located in "Central -Canada MSFT Datacenters(DC)".

Now question is, If we use "Azure Databricks",  Does all the data, and its controls is with in Azure or MSFT environment/DCs? I can see all IPs the domains resolve with in MSFT. Reference : https://learn.microsoft.com/en-us/azure/databricks/resources/ip-domain-region

But there are conflicting opinions like, " DataBricks is managed externally to Azure, even though I create workspace via Azure Databricks" - is this statement correct?

Please let me know . Thanks.

[SP_6721]

Hi @abin-bcgov 

If you’re using Azure Databricks in the Central Canada region, all your data and compute stays within your Azure subscription and the Microsoft datacenters in Canada.

The control plane, which handles things like notebooks and job management, is managed by Databricks. But it only deals with metadata and your actual data never leaves the region unless you explicitly configure it to. So, the idea that "Databricks is managed externally to Azure" may not be entirely accurate. While Databricks manages the backend, your data and workloads stay fully inside Azure and your chosen region.

[abin-bcgov]

@SP_6721 - Thanks for the reply. But a small confusion on your last statement, "While Databricks manages the backend, your data and workloads stay fully inside Azure and your chosen region". As per my analysis on Azure Databricks, none of artifacts leaves MSFT DC which I am designated or assigned. and Databricks as a Cloud provider or Solutions has no control on the deployment, data over "Azure Databricks".

[SP_6721 · Accepted solution]

@abin-bcgov 

Your data and compute workloads stay within your Azure subscription and the region you’ve chosen (Canada Central). That’s exactly how Azure Databricks is built to follow your organization’s compliance and data residency rules.

Just to clear up the “Databricks manages the backend” part, the control plane is handled by Databricks, but it's hosted within Microsoft Azure and shared across customers in your region. It doesn’t touch or access your actual data.

[abin-bcgov]

Thanks a ton, @SP_6721