does stream.stop() generates "ERROR: Query termination received for []" automatically?

[xhudik]

Whenever code contains stream.stop() in STDERR (in cluster logs) I get an error like:

ERROR: Query termination received for [id=b7e14d07-f8ad-4ae6-99de-8a7cbba89d86, runId=5c01fd71-2d93-48ca-a53c-5f46fab726ff]

No other message, even if I try to try-catch and get some error stack no success (always only this 1 line error). When I take a look on Log4j output, there is an info instead of error:

24/09/27 12:42:46 INFO DatabricksStreamingQueryListener: Query termination received for [id=b7e14d07-f8ad-4ae6-99de-8a7cbba89d86, runId=5c01fd71-2d93-48ca-a53c-5f46fab726ff]

I'm wondering:

• does stream.stop() generates "ERROR: Query termination received for []" automatically?
• Is ERROR: Query termination received for [] dangerous, or it is just ans info stream was closed?

[MuthuLakshmi · Accepted solution]

@xhudik 
does stream.stop() generates "ERROR: Query termination received for []" automatically?
Yes, this is generated when there is stream.stop() in stdderIs

ERROR: Query termination received for [] dangerous, or it is just ans info stream was closed?
The "Query termination received" message that is not preceded by an Exception is not a concern, since this message is simply triggered by streamingContext.stop(), which stops the execution of streams after processing all incoming data. This is considered a graceful stop scenario and a behavior by design for logs.