Databricks Community

Mridu

Hi Databricks Community I’m working on designing a practical Entity Resolution / Deduplication framework on Databricks, and I wanted to share a high‑level blueprint and learn from others who’ve tackled similar problems at scale.The use case is fairl...

Mridu

Hi @Ashwin_DSA -Thanks again for the detailed accelerator mapping — one follow‑up question I’m curious about from a delivery perspective:In regulated BFSI projects, how do teams typically decide where to draw the line between accelerator reuse vs cus...

Mridu

Hi @Ashwin_DSA — thank you for the detailed and thoughtful response, this is extremely helpful.I completely agree with your point about not reinventing the wheel and leveraging the Databricks Entity Resolution Solution Accelerators as a starting poin...

Mridu

This is expected behavior in Spark Structured Streaming, and the key point is that file streaming is not just driven by the checkpoint.Spark uses file metadata tracking at the source level, not only checkpoint state, to decide whether a file is “new”...

Databricks Community

User Stats

User Activity

Blueprint: Entity Resolution(Dedup & Golden Records)on Databricks — blocking, fuzzy scoring, MLflow

Re: Blueprint: Entity Resolution(Dedup & Golden Records)on Databricks — blocking, fuzzy scoring,

Re: Blueprint: Entity Resolution(Dedup & Golden Records)on Databricks — blocking, fuzzy scoring,

Re: Spark Streaming – Old file not processed with new checkpoint and new output path