end-to-end lineage using Unity Catalog

Phani1
Databricks MVP

Hi Team,

what is the Possibilities of getting end-to-end lineage using Unity Catalog

Regards,

Phanindra

MathieuDB
Databricks Employee
Databricks Employee

Hello Phanindra,

Inside Databricks ecosystem, you can easily get end-to-end lineage. The data lineage will be build automatically as you ingest and transform data inside the lakehouse. Here is a technical view on how this is achieved:

UC Deep Dive (1).png

On the other hand, if you were referring to end-to-end lineage from an enterprise perspective where Databricks is one component in your overall system, you could leverage a catalog of catalogs to get the full picture.

UC Deep Dive.png

Hope this helps,
Mathieu