Do you see a future where enterprise MDM platforms like Informatica MDM could be fully replaced ?

[rkhand14_ltm]

Do you see a future where enterprise MDM platforms like Informatica MDM could be fully replaced or reimagined on Databricks?

🚀 Databricks has evolved far beyond a data engineering platform. With its growing ecosystem of connectors, governance capabilities, AI features, and now Data Quality Monitoring, it continues to blur the lines between traditional platforms.

This got me thinking:

Do you see a future where enterprise MDM platforms like Informatica MDM could be fully replaced or reimagined on Databricks?

Can the Lakehouse architecture eventually handle:

✅ Master Data Management

✅ Golden Record Creation

✅ Match & Merge

✅ Survivorship Rules

✅ Hierarchies & Relationships

✅ Data Stewardship

Or do you believe specialized MDM platforms will always have a place in the enterprise landscape?

💬 Curious to hear from architects, data engineers, MDM specialists, and governance leaders.

What's your take?

#Databricks #MDM #DataGovernance #DataArchitecture #Lakehouse #DataEngineering #UnityCatalog #MasterDataManagement

[DoTA]

Good question, and one we've had to answer practically on a regulated (banking) data platform.

My take: Databricks + Unity Catalog now covers a surprising amount of "MDM-adjacent" plumbing - golden record storage, lineage, access governance, even survivorship logic if you're willing to write it yourself in DLT/Spark. Where it still falls short of purpose-built MDM (Informatica, Reltio, etc.):

1. Match & merge tooling - probabilistic/fuzzy matching with configurable stewardship workflows is far more mature in dedicated MDM products. You can build match/merge on Spark, but at that point you're rebuilding a product, not configuring one.

2. Stewardship UX - business users (not engineers) need a low-code interface to review merge candidates and override golden records. That's a workflow/UI problem more than a data problem, and it's not what the Lakehouse is optimized for.

3. Audit trail semantics - in banking specifically, "why was this the golden record" needs to be provable to regulators with a change-history model MDM tools ship out of the box. Doable on Delta (CDC + Unity Catalog audit logs) but you're assembling it yourself rather than getting it for free.

Where the lines genuinely blur: reference data management, hierarchies for less-regulated domains, and cases where "good enough" survivorship rules in a pipeline beat the cost of a full MDM license. For core customer/party golden records in a regulated industry, I'd still reach for a dedicated MDM tool with Databricks as the governed serving/analytics layer downstream.

Curious if anyone's tried building a stewardship UI on Databricks Apps - that's the piece I'd want solved before fully retiring MDM platforms.

[ShamenParis]

Hi @rkhand14_ltm ,

We are moving quickly toward a world where Databricks can handle almost the entire MDM workload natively through a Composable MDM approach.

Instead of buying a monolithic, black-box MDM, you can build it right where your data already lives:

• The Engine: Delta Live Tables (DLT) and Databricks AI easily handle the heavy lifting for Match & Merge, Survivorship rules, and generating Golden Records directly in the Lakehouse.

• The Application Layer (Stewardship): The biggest argument for legacy MDMs used to be the lack of a UI for Data Stewards. Now, with Databricks Apps, you can build custom, lightweight Data Stewardship UIs directly on top of your data (using your Lakebase as the foundation).

• Governance & Reporting: Databricks Dashboards and Unity Catalog handle all the data quality monitoring, lineage, and audit tracking out-of-the-box.

Why move your data out to a proprietary third-party tool when you can process the rules with DLT and serve the stewardship workflows via Databricks Apps? The Lakehouse is absolutely becoming the new center of gravity for master data.

Databricks continuously releases new tools and solutions to help us build solid frameworks, leveraging both native and open-source resources. For example, with the newly launched LTAP (Lake Transactional/Analytical Processing) architecture, we can now bring operational data directly into Databricks without the need for complex ETL pipelines.