Mridu
Databricks Partner

Hi @Ashwin_DSA — thank you for the detailed and thoughtful response, this is extremely helpful.

I completely agree with your point about not reinventing the wheel and leveraging the Databricks Entity Resolution Solution Accelerators as a starting point. The way you mapped the Customer ER / Product Matching / Public Sector ER accelerators to the modular stages I outlined is spot on and aligns very closely with the patterns I’ve seen as well.

My intent with this discussion was less about building a greenfield ER engine from scratch and more about:

  • understanding the core architectural patterns behind ER on Databricks, and
  • identifying which parts are best handled by out‑of‑the‑box accelerators versus where teams typically extend or adapt them for domain‑specific needs (especially in regulated environments).

The points you called out around:

  • multi‑key + semantic blocking,
  • graph‑style clustering,
  • deterministic survivorship rules,
  • MLflow‑managed models (including Zingg),
  • and modelling reviewer decisions as a labelled Delta table for both audit and retraining

are particularly useful and reinforce that the accelerators already encode many best practices.

I also like your recommendation of cloning the Customer Entity Resolution accelerator and adapting notebook‑by‑notebook, rather than maintaining a fully custom pipeline end‑to‑end — that’s a very pragmatic approach and likely reduces long‑term maintenance significantly.

Thanks again for taking the time to share this perspective — I’m sure this will be valuable for others exploring ER on Databricks as well.

Happy to mark this as the accepted solution 👍