Databricks (GCP) Cluster not resolving Hostname into IP address

[Sadam97]

we have #mongodb hosts that must be resolved to private internal loadbalancer ips ( of another cluster ), and that we are unable to add host aliases in the Databricks GKE cluster in order for the spark to be able to connect to a mongodb and resolve the mongodb hostname into ip address. Databricks workspace is in VPC as mongodb cluster and has subnet access as we are able to telnet the mongodb node ip from databricks noteboo

Getting error, Timed out after 30000 ms while waiting for a server that matches .

[NandiniN]

Hi @Sadam97 ,

As you are able to telnet to the MongoDB node's IP from the Databricks notebook, it means that network connectivity exists between Databricks and the MongoDB cluster. This confirms that the issue is likely related to DNS resolution or how MongoDB hosts are being addressed, rather than network access itself.

From a Databricks notebook or the GKE cluster, run a DNS query (e.g., nslookup or dig) to ensure the MongoDB hostnames resolve to the correct internal IP addresses. If DNS fails, you may need to configure a DNS server in your GKE cluster that can resolve the MongoDB private addresses.

Thanks!

[NandiniN]

However, there is something else that I would like you to quickly try - using a single-user cluster, and a shared access mode cluster. Do you observe the issue only in shared cluster? 

[NandiniN]

Also found this - https://community.databricks.com/t5/data-engineering/why-i-m-getting-connection-timeout-when-connect...