susmitsircar
New Contributor III

yeah I was aware of this one

spark.sql.streaming.stateStore.rocksdb.trackTotalNumberOfRows

https://spark.apache.org/docs/3.5.0/structured-streaming-programming-guide.html#performance-aspect-c...

Thanks for the clarification on the 

spark.databricks.rocksDB.verifyBeforeUpload false

 We will try this out marking it as a Solution for now. Thanks.