Connection closed by foreign host

[m-lopez]

Hi, I am setting up Unity Catalog in my environment, I have a workspace with 2 clusters, the older one (10.4 runtime and non-isolated shared mode) without Unity Catalog configuration and the new one with Unity Catalog enabled (13.3 runtime and shared mode). .

The problem I'm having is when I try to connect to Kafka (running on my aks cluster). I try a telnet connection from the old cluster and it works, but it doesn't work when I try from the new cluster with the Unity catalog setup.

This is what I got when I ran the telnet command from a notebook:

Trying 10.40.12.243...
Connected to 10.40.12.243.
Escape character is '^]'.
Connection closed by foreign host.

[Ayushi_Suthar]

Hi @m-lopez , hope you are doing well! 

Were you able to connect to Kafka before? Also, do you get the same issue if you run the telnet command against port 9092? 

As per the error message "Trying 10.40.12.243...Connected to 10.40.12.243. Escape character is '^]'.
Connection closed by foreign host." indicates that we were able to establish a connection to the Kafka broker, but the connection was immediately closed by the broker.

This could be due to several reasons:

1. The Kafka broker might not be running at the specified IP address or hostname. The error message you're seeing is often encountered when an invalid hostname or IP address is passed to the kafka.bootstrap.servers configuration

2. There may be a network security configuration issue. Kafka brokers might be configured to only accept connections from certain IP addresses, and the IP address you're connecting from may not be on that list.

3. Incorrect SSL configurations can also cause this issue.

Can you please check in all the above steps are correct?

Please let me know if this helps and leave a like if this information is useful, followups are appreciated.
Kudos
Ayushi