Access storage account with private endpoint

[icyflame92]

Hi, I need guidance on connecting Databricks (not VNET injected) to a storage account with Private Endpoint.

We have a client who created Databricks with (public ip and not VNET Injected). It’s using a managed VNET in the Databricks managed resource group and expose with public IP. We’re wondering if we still can make it connect to blob storage/ ADLS Gen2 over private endpoints. 

We want use Oauth2 with Service Principal with Storage Blob Data Contributor as role set on the blob storage/ ADLS Gen2 and want to mount in Workspace with Service Principal credentials. In customer Workspace, UC is not activated, no possibility via UC access connector.

So can we use this workspace setup (public ip and not VNET Injected) to access storage with private endpoint use mounting?

Thanks in advance!

[rudyevers · Accepted solution]

 No this is not possible because the workspace is not part of the virtual network and since than can not access the storage over it's private endpoint. It is all mentioned in de documentation:

https://www.databricks.com/blog/2020/02/28/securely-accessing-azure-data-sources-from-azure-databric...

So, if you are not able to integrate the workspace in to you vnet there are some workarounds. You can add a storage account to a application gateway, so it is protected from the internet but still publicly available. BUT, if you don't have an application gateway it's an expensive solution. My advice is to follow the best practices and integrated databricks into the virtual network.

[icyflame92]

From our point of view, it is definitely worth it best practice to follow instead of starting workarounds. Also the infra is not in our hands, hence is no justification to start a workaround with Application Gateway. Thank you @rudyevers