MetaData Framework for Multi-Level Silver Layer PK/FK Creation
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-06-2026 07:02 AM
Hi Team,
Our source data originates from MongoDB as hierarchical JSON and is ingested into our Silver layer, where it is already normalized into tabular structures. However, primary and foreign key constraints are not enforced during ingestion. We must establish and maintain PK/FK relational integrity post-loading through an automated post-processing step.The Challenge:The source system features a deep, multi-level parent-child tree hierarchy (e.g., emp -> emp_addr -> emp_addr_ofc). We require a metadata-driven modeling framework and script execution strategy to dynamically generate surrogate keys, resolve parent-child references top-down, and enforce relational integrity across all levels.
Please find below screen shot for your reference.