- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
4 weeks ago
Hi everyone,
I’m currently working on a PoC to build a data dictionary in Databricks, and one of the requirements is to retrieve lineage information programmatically.
I was advised that Databricks provides Unity Catalog Lineage APIs that can retrieve table-level and column-level lineage, following the Unity Catalog permission model and without requiring direct access to the system.access schema.
I was provided with an example endpoint:
/api/2.1/unity-catalog/lineage/table-lineages
However, after searching the current Databricks documentation, I cannot find this endpoint or any official documentation for a table-lineages REST API.
The situation is that our parent company does not allow us to have direct access to the system.access schema for security and governance reasons. Therefore, querying system.access.table_lineage is not an option for our PoC.
My goal is to build a data dictionary and enrich it with lineage information, so I’m looking for a supported REST API that would allow me to retrieve table-level lineage without accessing system.access.table_lineage directly.
Could anyone clarify:
Is /api/2.1/unity-catalog/lineage/table-lineages a supported/public Databricks REST API?
If so, where can I find the official documentation?
Is there another supported REST API to retrieve table-level lineage without querying system.access.table_lineage?
Is column-level lineage also available through a supported API?
Any guidance or documentation would be greatly appreciated.
Thanks!