User16826994223
Databricks Employee
Databricks Employee

If you have configured your Structured Streaming query to use RocksDB as the state store, you can now get better visibility into the performance of RocksDB, with detailed metrics on get/put latencies, compaction latencies, cache hits, and so on. These metrics are available through the 

StreamingQueryProgress and StreamingQueryListener

 APIs for monitoring a streaming query.

More Info -https://docs.microsoft.com/en-gb/azure/databricks/spark/latest/structured-streaming/production#rocksdb-metrics

View solution in original post