topic Re: UnknownHostException in Unity Catalog in Data Governance

UnknownHostException in Unity Catalog

ossinova — Mon, 17 Oct 2022 13:39:24 GMT

I have a few catalogs in databricks (using Unity Catalog). Each catalog (DEV, TEST, PROD) having its own external location (ADLSv2) and being tied to its own dbx workspace. Seeing how unity unifies the data, access policies and the metastore I would assume I could query data in PROD from DEV and vice versa.

However, I am getting this error when trying to do so:

-- in dev environment
USE CATALOG prod;
SELECT * FROM bronze.demo;

>> Error in SQL statement: ExecutionException: Status code: -1 error code: null error message: Cannot resolve hostname: xxxxPRODxxx.dfs.core.windows.net

java.net.UnknownHostException: xxxxPRODxxx.dfs.core.windows.net

Re: UnknownHostException in Unity Catalog

karthik_p — Mon, 17 Oct 2022 14:20:57 GMT

@Oscar Dyremyhr It looks issue is not specific to unity catalog, but DNS resolution, can you try below steps from your unity catalog configured cluster by following below steps in article

UnknownHostException on cluster launch - Azure Databricks | Microsoft Learn

Re: UnknownHostException in Unity Catalog

ossinova — Mon, 17 Oct 2022 14:40:50 GMT

@karthik p The issue is not upon cluster launch, although it might be related. I am still able to access data from DEV. I will take a look.

Re: UnknownHostException in Unity Catalog

ossinova — Mon, 24 Oct 2022 09:43:42 GMT

@Debayan Mukherjee Thanks for the feedback. This solved my issue. Had to add some private endpoints to the underlying storage accounts in where the catalog was referencing data from.

Re: UnknownHostException in Unity Catalog

Debayan — Tue, 18 Oct 2022 06:33:46 GMT

Hi @Oscar Dyremyhr , looking at the error it looks like it is not able to resolve the hostname, could you please try to nslookup the hostname and see if it is getting resolved? Also, what is used in your connection string? ("account" can be used)