cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Databricks (GCP) Cluster not resolving Hostname into IP address

Sadam97
New Contributor

we have #mongodb hosts that must be resolved to private internal loadbalancer ips ( of another cluster ), and that we are unable to add host aliases in the Databricks GKE cluster in order for the spark to be able to connect to a mongodb and resolve the mongodb hostname into ip address. Databricks workspace is in VPC as mongodb cluster and has subnet access as we are able to telnet the mongodb node ip from databricks noteboo

Getting error, Timed out after 30000 ms while waiting for a server that matches .

2 REPLIES 2

NandiniN
Databricks Employee
Databricks Employee

Hi @Sadam97 ,

As you are able to telnet to the MongoDB node's IP from the Databricks notebook, it means that network connectivity exists between Databricks and the MongoDB cluster. This confirms that the issue is likely related to DNS resolution or how MongoDB hosts are being addressed, rather than network access itself.

From a Databricks notebook or the GKE cluster, run a DNS query (e.g., nslookup or dig) to ensure the MongoDB hostnames resolve to the correct internal IP addresses. If DNS fails, you may need to configure a DNS server in your GKE cluster that can resolve the MongoDB private addresses.

Thanks!

NandiniN
Databricks Employee
Databricks Employee

However, there is something else that I would like you to quickly try - using a single-user cluster, and a shared access mode cluster. Do you observe the issue only in shared cluster? 

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group