Data quality Lineage Root cause analysis

Niyojit
Databricks Partner

Hi everyone,

I'm working on a client use case where they want to visualize data quality together with data lineage.

My current approach is to combine:

  • Unity Catalog Data Lineage for upstream/downstream dependencies
  • Lakehouse Monitoring (or other data monitoring capabilities)
  • DQX (Data Quality Expectations) library for table-level data quality checks

The idea is to display the lineage graph and, for each table in the lineage, show the corresponding data quality metrics/check results (e.g., passed/failed expectations, freshness, completeness, etc.) directly alongside the table node.

I'm wondering:

  • Has anyone implemented something similar in Databricks?
  • Is it possible to enrich the lineage graph with DQX or monitoring results?
  • Are there recommended APIs, system tables, or best practices for achieving this?
  • Or is there a better architecture or approach to solve this use case?

I'd appreciate any suggestions or insights from the community.

Thanks in advance!

Niyojit