jose_gonzalez
Databricks Employee
Databricks Employee

Hi @itay k​ ,

You will need to take a look at the Progress Reporter. This will show the Micro-batch JSON metrics. For example, the metric called "numInputRows" which will display the number of input rows that it processed for the micro-batch. You will find these metrics in the driver logs --> log4j

In addition, the following article will show how are these Streaming metrics mean and how to access/view them https://databricks.com/blog/2020/07/29/a-look-at-the-new-structured-streaming-ui-in-apache-spark-3-0...