topic Re: Not able to catch structured streaming exception in Data Engineering

Not able to catch structured streaming exception

415963 — Tue, 25 Jul 2023 14:49:16 GMT

I would like to catch and handle an exception in a structured streaming job.

The databricks notebook still displays the exception, regardless of added exception handling (see attached screenshot)

I guess that the exception is displayed by the cell output. Is it possible to ignore the stream display in the cell output?

Re: Not able to catch structured streaming exception

Debayan — Tue, 25 Jul 2023 14:58:50 GMT

Hi,

Could you please expand the error section and paste it here?

Also, please tag @Debayan with your next comment which will notify me. Thanks!

Re: Not able to catch structured streaming exception

415963 — Tue, 25 Jul 2023 15:28:57 GMT

@Debayan

The notebook fails, even though I would like to catch that exception and return a different message to the caller. Hence, the notebook should end successfully.

Re: Not able to catch structured streaming exception

Debayan — Wed, 26 Jul 2023 15:21:15 GMT

Hi, I understand, could you please also provide the last line of the error after scrolling down in the notebook cell?