AbfsRestOperationException when adding privatelink.dfs.core.windows.net

[fabian564]

Hey Databricks forum,

Have been searching a lot, but can't find a solution. I have the following setup:

- a vnet connected to the databricks workspace with
   - public-subnet (deligated to Microsoft.Databricks/workspaces) and a NSG
   - private-subnet (deligated to Microsoft.Databricks/workspaces) and a NSG
   - private endpoints subnet

- Azure Data Lake Storage Gen2 with Hierarchical Namespace enabled
  - has a container 'metastore'
  - Public network access is disabled
  - Added private endpoint
      - Target sub-resource is dfs
      - to the private endpoints subnet
      - added dns zone privatelink.dfs.core.windows.net with a A record [name of storage resource] and also a vnet link to the vnet.

- Unity-catalog-access-connector with managed identity
 - with RBAC permissions with Storage Blob Data Contributor on the storage.

If I add a new external location with the following input:
- Storage credential - (refrenced with the ResourceId of the Unity-catalog-access-connector)
- abfss://metastore@[name of storage resource].privatelink.dfs.core.windows.net

I recieve the message:

> Failed to access cloud storage: [AbfsRestOperationException] () exceptionTraceId=ff1075e9-00d9-44b6-a602-9d7c19fbae9b

When I give the storage a public-ip and set 'staimzdatabricks.dfs.core.windows.net' it succeeds.

If I search for this exception, I get the response this could be network or permission related.
I run from the Databricks workspace a compute a nslookup for [name of storage resource].privatelink.dfs.core.windows.net:

>Server: 168.63.129.16
> Address: 168.63.129.16#53
>
> Non-authoritative answer:
> Name: [name of storage resource].privatelink.dfs.core.windows.net
> Address: 10.1.1.4

Do you have an I idea what I am missing here?

[szymon_dybczak]

Hi @fabian564 ,

Try to add also blob private endpoint. I think it's also needed for some metadata operations.

[fabian564]

Hi @szymon_dybczak ,

I added also the blob private endpoint. When you add a new Databricks external location there is only 1 URL field, which defines for `dfs` url, also tried with abfss://metastore@[name of storage resource].privatelink.blob.core.windows.net/ same result (Failed to access cloud storage: [AbfsRestOperationException]).

I see this suggestion more often, I think this is referring to the NCC configuration in Databricks, this is for the Databricks Serverless SQL Compute, there you can add private endpoint rules and is given a blob and dfs private endpoint 🙂