Hi,

Did you check the logs for the micro-batch metrics? these metrics will help you to identify the number of records you process and the time it took to process the data. This article shows the metrics that are capture and how you can use them to know how is your streaming job working. Link https://www.waitingforcode.com/apache-spark-structured-streaming/query-metrics-apache-spark-structur...

Thank you