topic Re: Spark streaming query stops after code exception in notebook since 14.3 in Get Started Discussions

Spark streaming query stops after code exception in notebook since 14.3

jcozar — Thu, 04 Apr 2024 14:18:53 GMT

Hi!

I am experiencing something that I cannot find in the documentation: in databricks, using the databricks runtime 13.X, when I start a streaming query (using .start method), it creates a new query and while it is running I can execute other code in other cells of the notebook. Even if that cells raises an exception, the query continues until I interrupt the cell execution or use the stop method for the query.

However, since databricks runtime 14.3 (I don't know if it happens in 14.2 and 14.1), if the code run in a new cell causes an exception the query is stopped. Is it configurable? Where is that documented?

Thank you!

Re: Spark streaming query stops after code exception in notebook since 14.3

Lakshay — Thu, 04 Apr 2024 15:36:42 GMT

@jcozar , This looks like a potential bug. If you can raise a support case, that would be helpful

Re: Spark streaming query stops after code exception in notebook since 14.3

jcozar — Thu, 04 Apr 2024 15:51:45 GMT

@Lakshay thank you, I also think so! If you don't mind, where can I raise a support case?

Thank you!

Re: Spark streaming query stops after code exception in notebook since 14.3

Lakshay — Thu, 04 Apr 2024 15:59:26 GMT

You can use the help portal: https://help.databricks.com/s/

Re: Spark streaming query stops after code exception in notebook since 14.3

jcozar — Sun, 07 Apr 2024 09:24:04 GMT

It seems it has been solved! 🙂