01-30-2023 01:01 AM
According to the documentation you can monitor a spark structure stream job using QueryExecutionListener. However I cannot find it.
https://docs.databricks.com/structured-streaming/stream-monitoring.html#language-python
02-23-2023 09:58 AM
Which DBR version are you using? also, can you share some code snippet on how you are using the QueryExecutionListener?
never-displayed