Hi @susmitsircar, spark.databricks.rocksDB.verifyBeforeUpload config determines whether a verification check should be conducted prior to uploading data to RocksDB. The default value is true. Since the SST files are lost, disabling the above config will help to bypass the error.

Regarding the other config, spark.sql.streaming.stateStore.rocksdb.trackTotalNumberOfRows, setting it to false causes Spark to skip tracking the number of rows, which can result in faster write operations and improved performance, especially in high-volume or heavily stateful streaming workloads.

Hope this helps!