- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-15-2026 10:43 PM
Hi @Ashwin_DSA — thank you for the detailed and thoughtful response, this is extremely helpful.
I completely agree with your point about not reinventing the wheel and leveraging the Databricks Entity Resolution Solution Accelerators as a starting point. The way you mapped the Customer ER / Product Matching / Public Sector ER accelerators to the modular stages I outlined is spot on and aligns very closely with the patterns I’ve seen as well.
My intent with this discussion was less about building a greenfield ER engine from scratch and more about:
- understanding the core architectural patterns behind ER on Databricks, and
- identifying which parts are best handled by out‑of‑the‑box accelerators versus where teams typically extend or adapt them for domain‑specific needs (especially in regulated environments).
The points you called out around:
- multi‑key + semantic blocking,
- graph‑style clustering,
- deterministic survivorship rules,
- MLflow‑managed models (including Zingg),
- and modelling reviewer decisions as a labelled Delta table for both audit and retraining
are particularly useful and reinforce that the accelerators already encode many best practices.
I also like your recommendation of cloning the Customer Entity Resolution accelerator and adapting notebook‑by‑notebook, rather than maintaining a fully custom pipeline end‑to‑end — that’s a very pragmatic approach and likely reduces long‑term maintenance significantly.
Thanks again for taking the time to share this perspective — I’m sure this will be valuable for others exploring ER on Databricks as well.
Happy to mark this as the accepted solution 👍