cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Machine Learning
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
elementalM
by • New Contributor III
  • 1337 Views
  • 4 replies
  • 4 kudos

Catch-up Structured Stream hangs on last step of write job to delta sync using toTable

I'm running databricks version 10.4 on gcp. I'm running a structured stream trying to process historical files in a delta table on gcp cloud storage. This source delta table is big but maintained with OPTIMIZE.The stream repartitions which seems to b...

image
Machine Learning
Reply
  • 1337 Views
  • 4 replies
  • 4 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 4 kudos

Hi @Dwight Branscombe​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you....

  • 4 kudos
3 More Replies
User16826994223
by • Honored Contributor III
  • 1948 Views
  • 1 replies
  • 1 kudos

Resolved! Can I get detailed metrics of the RocksDB that I have used in one of my streams ?

Machine Learning
Reply
  • 1948 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
User16826994223
Honored Contributor III
  • 1 kudos

If you have configured your Structured Streaming query to use RocksDB as the state store, you can now get better visibility into the performance of RocksDB, with detailed metrics on get/put latencies, compaction latencies, cache hits, and so on. Thes...

  • 1 kudos
Labels