Databricks Community

mriccardi · 06-02-2023

Hi Everyone!Today 4 streaming jobs started to fail out of nowhere due to: StreamingQueryException: [STREAM_FAILED] Query [id = ####, runId = ####] terminated with exception: dbfs:/mnt/path/my_table/sources/0/0 doesn't exist (latestId: 8, compactInter...

mriccardi · 12-01-2022

Hello,We are experiencing an error with one Structured Streaming Job that we have, that basically the checkpoint gets corrupted and we are unable to continue with the execution.I've checked the errors and this happens when it triggers an autocompact,...

mriccardi · 07-26-2022

Hello everyone!We are currently facing an issue with a stream that is not updating new data since the 20 of July.We've validated and bronze table has data that silver doesn't have.Also seeing the logs the silver stream is running but writing 0 files....

mriccardi · 07-26-2022

Also the trigger is configured to run once, but when we start the job it never ends, it keeps in an endless loop.

Databricks Community

User Stats

User Activity

Spark Streaming: Checkpoint corrupted

Structured Streaming Checkpoint corrupted.

Spark streaming: Checkpoint not recognising new data

Re: Spark streaming: Checkpoint not recognising new data