Exposing Unity Catalog lineage schema

[squarshie]

Have been informed by a vendor we are using for Data Observability that the Unity Catalog lineage can be accessed via the systems schema. This is not enabled by default and we will need to execute a PUT request to the endpoint below. I am unable to find any documentation for this.

api/2.0/unity-catalog/metastores/<metastore_id>/systemschemas/lineage

Is any one familiar with this API call and have details on what exactly this call is going to do?

[Debayan]

Hi,

UC APIs are under API 2.1. You can get all the API details here: https://docs.databricks.com/dev-tools/api/latest/index.html and

https://docs.databricks.com/api-explorer/workspace/catalogs/list

I do not think there is an API as mentioned above.

Please tag @Debayan​ with your next comment so that I will get notified. Thank you!

[squarshie · Accepted solution]

@Debayan Mukherjee​ 

Found out this is a preview feature and hence why you cannot find any documentation on the api website. Feature exposes a lineage table in the system schema so you can programmatically query lineage data. While you can access linage via the API, our specific use case required being able to query this information directly from a table.

[Debayan]

Hi, Yes, this feature is on private preview.

[Anonymous]

Hi @Stephen Quarshie​ 

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. 

We'd love to hear from you.

Thanks!