UnknownHostException in Unity Catalog

[ossinova]

I have a few catalogs in databricks (using Unity Catalog). Each catalog (DEV, TEST, PROD) having its own external location (ADLSv2) and being tied to its own dbx workspace. Seeing how unity unifies the data, access policies and the metastore I would assume I could query data in PROD from DEV and vice versa.

However, I am getting this error when trying to do so:

-- in dev environment
USE CATALOG prod;
SELECT * FROM bronze.demo;

>> Error in SQL statement: ExecutionException: Status code: -1 error code: null error message: Cannot resolve hostname: xxxxPRODxxx.dfs.core.windows.net

java.net.UnknownHostException: xxxxPRODxxx.dfs.core.windows.net

[karthik_p]

@Oscar Dyremyhr​ It looks issue is not specific to unity catalog, but DNS resolution, can you try below steps from your unity catalog configured cluster by following below steps in article

UnknownHostException on cluster launch - Azure Databricks | Microsoft Learn

[ossinova]

@karthik p​ The issue is not upon cluster launch, although it might be related. I am still able to access data from DEV. I will take a look.

[Debayan]

Hi @Oscar Dyremyhr​ , looking at the error it looks like it is not able to resolve the hostname, could you please try to nslookup the hostname and see if it is getting resolved? Also, what is used in your connection string? ("account" can be used)

[ossinova]

@Debayan Mukherjee​ Thanks for the feedback. This solved my issue. Had to add some private endpoints to the underlying storage accounts in where the catalog was referencing data from.