end-to-end lineage using Unity Catalog

Phani1 — Thu, 25 Jan 2024 06:58:52 GMT

Hi Team,

what is the Possibilities of getting end-to-end lineage using Unity Catalog

Regards,

Phanindra

Re: end-to-end lineage using Unity Catalog

MathieuDB — Thu, 28 Nov 2024 18:31:50 GMT

Hello Phanindra,

Inside Databricks ecosystem, you can easily get end-to-end lineage. The data lineage will be build automatically as you ingest and transform data inside the lakehouse. Here is a technical view on how this is achieved:

On the other hand, if you were referring to end-to-end lineage from an enterprise perspective where Databricks is one component in your overall system, you could leverage a catalog of catalogs to get the full picture.

Hope this helps,
Mathieu

topic Re: end-to-end lineage using Unity Catalog in Get Started Discussions

end-to-end lineage using Unity Catalog

Re: end-to-end lineage using Unity Catalog