topic Re: Dataplex integration with Unity Catalog in Data Governance

Dataplex integration with Unity Catalog

ShankarM — Tue, 21 Jan 2025 07:42:59 GMT

Hi,

I want to import Databricks metadata information from unity catalog including lineage into GCP Dataplex governance solution. Can you let me know the how this can be achieved.

Similar to Purview and Unity Catalog integration, are there APIs to push information from Unity Catalog into Dataplex.

Thanks

Shankar

Re: Dataplex integration with Unity Catalog

Walter_C — Tue, 21 Jan 2025 12:45:11 GMT

To import Databricks metadata information from Unity Catalog, including lineage, into GCP Dataplex, you can use the Unity Catalog APIs to extract the necessary metadata and lineage information and then push this data into Dataplex using its APIs.

Here are the steps you can follow:

Extract Metadata and Lineage from Unity Catalog:
- Use the Unity Catalog REST APIs to retrieve metadata and lineage information. You can find the API documentation and endpoints for Unity Catalog in the Databricks documentation. The relevant APIs include those for listing catalogs, schemas, tables, and retrieving lineage information.
Transform the Data:
- Transform the extracted metadata and lineage information into the format required by GCP Dataplex. This may involve converting the data into JSON or another suitable format that Dataplex can ingest.
Push Data to GCP Dataplex:
- Use the Dataplex APIs to push the transformed metadata and lineage information into Dataplex. The Dataplex documentation provides details on how to use their APIs for metadata management.

Currently, there is no direct integration between Unity Catalog and GCP Dataplex similar to the integration between Purview and Unity Catalog. Therefore, you will need to use the APIs provided by both platforms to achieve this integration manually.

Re: Dataplex integration with Unity Catalog

ShankarM — Tue, 21 Jan 2025 13:23:14 GMT

Thanks Walter for the information. Can you share the links for the APIs.

Re: Dataplex integration with Unity Catalog

ShankarM — Tue, 21 Jan 2025 13:38:21 GMT

Hi Walter

Is it possible to import metadata from any governance/catalog tool into unity catalog or it is only push mechanism from unity catalog into dataplex/purview?

Re: Dataplex integration with Unity Catalog

sujand — Thu, 23 Apr 2026 16:35:34 GMT

Hi Walter,

Is it possible to migrate Databricks Unity Catalog data from Databricks to Google Dataplex without using API calls? What's the efficient way to move Unity Catalog data from Databricks to Google Dataplex? Please explain the steps. Thank you