topic DLT Online Table with VNnet Enable on Blob Storage Get 403 Issue in Get Started Discussions

DLT Online Table with VNnet Enable on Blob Storage Get 403 Issue

samlexrod — Thu, 18 Jul 2024 23:24:05 GMT

I am trying to create an online table in a Unity catalog. However, I get a GET, 403 error.

DataPlaneException: Failed to start the DLT service on cluster . Please check the stack trace below or driver logs for more details. com.databricks.pipelines.execution.service.UCContextInitializationException: Failed to initialize the UCContext com.databricks.pipelines.common.CustomException: [DLT ERROR CODE: EXECUTION_SERVICE_STARTUP_FAILURE.STORAGE_PERMISSION_ISSUE] Operation failed: "This request is not authorized to perform this operation.", 403, GET

This error only happens when I set my ADLS Gen 2 Networking Public network access settings to Enabled from selected virtual networks and IP addresses.
The online table gets created When I Enable it from all networks.

I have the correct access control using the unity-catalog-access-connector with Storage Blob Data Contributor.

My Databricks workspace is set up in a VNet with two subnets: the private and the public. These two subnets are white-listed in the network settings of my ADSL Gen2 in the Virtual Networks section of the Networking settings.

Yet, the only way I can set up the DLT Online Table is by setting my Blob storage to Enable it form all networks. How do I do this without Enabling it to all networks?

Re: DLT Online Table with VNnet Enable on Blob Storage Get 403 Issue

samlexrod — Fri, 19 Jul 2024 17:41:15 GMT

Hi @Retired_mod, Thank you for the fast response.

I believe I have whitelisted the network correctly. I managed to create the metastore and assign to the workspace. I also have the ability to create tables in the ADLS Gen2 unitycatalog container assigned to the metastore. The only thing that does not work is creating the online table.

Here is a screenshot of the VNet whitelisting. Perhaps the creation of the online table is not using the unity connector to access the resource. I have included a screenshot of the IAM role assigned to the blob storage.

Re: DLT Online Table with VNnet Enable on Blob Storage Get 403 Issue

samlexrod — Fri, 19 Jul 2024 19:39:18 GMT

I figured it out. It was because of the Network Connectivity Configurations. I did not have one setup with a private endpoint connection to the ADLS Gen2. I followed the instructions here: https://learn.microsoft.com/en-us/azure/databricks/security/network/serverless-network-security/serverless-private-link and it is now working with the VNet integrated.

Thank you @Retired_mod for your time.