LiresaFerizaj
Contributor

Great question. Here's what has worked for us.

Conformation: We do entity resolution in Silver (a crosswalk mapping each source system's ID to one enterprise key, with survivorship rules), then publish conformed dimensions once as shared Gold tables that every mart references. For SCD2 at scale, AUTO CDC in Lakeflow Declarative Pipelines handles history and out-of-order data well. We also expose a "current" view, since most consumers only need the latest version.

Unity Catalog: Conformed dimensions live in a dedicated catalog owned by a central team, with read-only access for marts and PII protected via column masks. Domain teams can extend but not modify: they build their own extension tables or rollups keyed off the conformed key, so nobody is blocked.

Schema evolution: Treat views as the contract. Additive changes go in place; breaking changes ship as a new versioned view with a deprecation window. UC lineage shows downstream impact before any change. For contract testing, we use pipeline expectations on the provider side (unique keys, one current row per business key) and schema checks in CI on the consumer side.

Lessons: Only conform what's truly shared, start with the dimensions causing the most reconciliation pain (usually Customer), and give each one a clear business owner. Most conformity issues turn out to be definition disputes, not technical ones.

Liresa Ferizaj