- 475 Views
- 9 replies
- 6 kudos
Streaming Failure Models: Why "It Didn't Crash" Is the Worst Outcome
Most Databricks streaming failures don't look dramatic.No cluster termination. No red wall of errors. The UI says RUNNING — and your customers start reporting nonsense.I wrote about the incident that changed how we think about streaming jobs on share...
- 475 Views
- 9 replies
- 6 kudos
- 6 kudos
Completely agree, production war stories are worth more than any documentation. I’ve eaten enough teeth on production data lake issues to write my own chapter on what can go wrong, whether that’s deploying Databricks in financial institutions or bein...
- 6 kudos
- 258 Views
- 1 replies
- 2 kudos
Multi-Task on a Shared Cluster — Why That's Also Not Enough
Part 2 of 3 — Databricks Streaming ArchitectureThe instinct after Part 1 was obvious.If running eight queries in one task means one failure can hide while others keep running — split them into multiple tasks. Separate concerns. Give each component it...
- 258 Views
- 1 replies
- 2 kudos
- 2 kudos
Part 1: Streaming Failure Models: Why "It Didn't Crash" Is the Worst OutcomePart 3: One Cluster per Task — Proven, Ready, and Waiting
- 2 kudos