Power BI - DBX Unity catalog Lineage Syncer

[Jeffdenzel]

Hi folks, 

So I have recently, as a side project been working on a tool, Link to my github: https://github.com/JeffDenzel/lineage-syncer.

The problem

The reason for it is that DataBricks allows you to put external data assets into your UC. However, there's not really an automatic sync between Power BI's semantic models and DBX's UC. So it's pretty annoying that everything has to be done manually.

I first checked out if there are different tools (I know a lot of people use Purview) but from what I can find Purview only allows you to scan Both the UC and the PBI service seperately. It doesn't connect them automatically.

I think the power of the UC comes from being a (primarily technical-oriented) governance tool. So from my PoV as a data engineer I would like to know, if/when I am changing gold products, then what downstream effects do I have on Reports/Models.

Because honestly, I may have broken some reports by renaming columns to a new standard 🙂 and then I have to deal with angry analysts.

The solution

A tool that automatically scans your power bi workspace, normalizes the response and then pushes it into unity catalog!

[Pavan_suresh]

This is great work, Jeff!

I have had my own share of pain with Power BI reports breaking because of column/name changes, so this problem feels very real.

Linking Power BI semantic models back to Unity Catalog lineage sounds like a very useful direction.
Lineage is becoming more important than ever