Thanks for your reply, and it's a super catch, now I am able to connect the dots.

I’ve confirmed that the DBR version is 9.1. I wanted to discuss some configurations related to RocksDB and performance optimizations. Specifically, I am considering adjusting the following settings:

spark.sql.streaming.stateStore.rocksdb.trackTotalNumberOfRows false​

This is currently set to true by default, but I am contemplating turning it off for better performance, especially since tracking the number of rows in the state store adds overhead on write operations. From your experience, is disabling this setting beneficial in terms of performance, especially when dealing with large state sizes? I understand that turning it off will report numTotalStateRows as 0, but it should help improve throughput.

spark.databricks.rocksDB.verifyBeforeUpload false

This I didn't found any documentation (not public made by databricks), so keen to know how it's going to impact, and what it does.