Most Databricks streaming failures don't look dramatic.

No cluster termination. No red wall of errors. The UI says RUNNING — and your customers start reporting nonsense.

I wrote about the incident that changed how we think about streaming jobs on shared clusters:

- Why query-scoped failures are more dangerous than driver-scoped ones
- How query.awaitTermination() on each stream individually caused us to miss a silent failure for 12 minutes
- Why don't continuous jobs save you if the JVM never fails in the first place
- The one-line fix (awaitAnyTermination) that stopped the lying — and why it's still a band-aid

Full Post at Medium or Blog

Part 2 (multi-task on a shared cluster — why that's also not enough) coming soon.

I am always happy to share/Learn about Production Insights!!!