topic Azure Event Hub throws Timeout Exceptio: Timed out waiting for a node assignment. Call: describeTopi in Data Engineering

Azure Event Hub throws Timeout Exceptio: Timed out waiting for a node assignment. Call: describeTopi

Ivan_Pyrog — Fri, 24 Jan 2025 11:18:17 GMT

Hello team,

We are researching the streaming capabilities of our data platform and currently in need of reading data from EVH ( event hub) with our Databricks notebooks. Unfortunately there seems to be an error somewhere due to Timeout Exception: Timed out waiting for a node assignment. Call: describe Topics.

Can you please help us tackle it?

The connections were assembled as per article https://learn.microsoft.com/en-us/azure/databricks/delta-live-tables/event-hubs ;
EVH networking is set to Public access
DBR runtime is set as "spark version": "15.4.x-scala2.12"

KAFKA_OPTIONS = { "kafka.bootstrap.servers" : f"{EH_NAMESPACE}.servicebus.windows.net:9093", "subscribe" : EH_NAME, "kafka.sasl.mechanism" : "PLAIN", "kafka.security.protocol" : "SASL_SSL", "kafka.group.id" : "databricks-streaming-eh-kelvin", "kafka.sasl.jaas.config" : f"kafkashaded.org.apache.kafka.common.security.plain.PlainLoginModule required username=\"$ConnectionString\" password=\"{EH_CONN_STR}\";", "kafka.request.timeout.ms" : 60000, # https://learn.microsoft.com/en-us/azure/event-hubs/apache-kafka-configurations#consumer-configurations-only "kafka.session.timeout.ms" : 300000, "maxOffsetsPerTrigger" : 4000, "failOnDataLoss" : True, "startingOffsets" : "latest" }

df = (spark.readStream .format("kafka") .options(**KAFKA_OPTIONS) .load())

Generates the proper schema, so some sort of handshake happened between EVH and DBR

However if I try to display my def.:

display(df) java.util.concurrent.ExecutionException: kafkashaded.org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: describeTopics Caused by: kafkashaded.org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: describeTopics

Re: Azure Event Hub throws Timeout Exceptio: Timed out waiting for a node assignment. Call: describe

Sidhant07 — Thu, 30 Jan 2025 07:55:10 GMT

Hi @Ivan_Pyrog ,

java.util.concurrent.ExecutionException: kafkashaded.org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: describeTopics <p><strong>Caused by: kafkashaded.org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: describeTopics</strong></p><p> </p>

Looking at the above error it is seems that the above errors are occuring due to no network connectivity between Databricks and Kafka cluster. Please make sure that telnet or nc commands should be successful before trying to execute any code to Kafka cluster.

for example

%sh telnet b-2.scram.cxdevorderbrokerstr<>xyz 14002

Also, make sure to enable SSL connections if Kafka cluster requires SSL connection. Please refer below link for SSL connection code snippet:

[+] https://docs.databricks.com/en/structured-streaming/kafka.html

Re: Azure Event Hub throws Timeout Exceptio: Timed out waiting for a node assignment. Call: describe

VZLA — Tue, 17 Jun 2025 14:00:39 GMT

@Ivan_Pyrog what's the full error message as per the Spark Driver log and what is your Kafka Broker version? I suspect you may be actually be hitting an incompatible of client-server.