Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
Showing results for 
Search instead for 
Did you mean: 

How to Export lineage data directly from unity catalog without using system tables

New Contributor

I have been trying to check if there is any direct way to export lineage hierarchy data in data bricks.

I have tried to build a workaround solution by accessing system tables as per this link:

Monitor usage with system tables - Azure Databricks | Microsoft Learn

But want to check if it can be achieved using interface itself instead?


Community Manager
Community Manager

Hi @Mahesh_Yadav, You can capture lineage data directly from Unity Catalog in Databricks without relying on system tables.

Here’s how you can achieve this:

  1. Using Unity Catalog:

    • Go to your Databricks landing page.
    • Click “New” in the sidebar and select “Notebook” from the menu.
    • Enter a name for the notebook and choose “SQL” as the default language.
    • Select a cluster with access to Unity Catalog.
    • Click “Create.”
    • Unity Catalog allows you to capture runtime data lineage across queries run on Databricks. Lineage is supported for all languages and is captured down to the column level. It includes notebooks, workflows, and dashboards related to the query1.
  2. Benefits of Unity Catalog:

Give this approach a try, and it should help you achieve your goal without relying on system tables! 😊🚀

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!