Hi folks,
So I have recently, as a side project been working on a tool, Link to my github: https://github.com/JeffDenzel/lineage-syncer.
The problem
The reason for it is that DataBricks allows you to put external data assets into your UC. However, there's not really an automatic sync between Power BI's semantic models and DBX's UC. So it's pretty annoying that everything has to be done manually.
I first checked out if there are different tools (I know a lot of people use Purview) but from what I can find Purview only allows you to scan Both the UC and the PBI service seperately. It doesn't connect them automatically.
I think the power of the UC comes from being a (primarily technical-oriented) governance tool. So from my PoV as a data engineer I would like to know, if/when I am changing gold products, then what downstream effects do I have on Reports/Models.
Because honestly, I may have broken some reports by renaming columns to a new standard 🙂 and then I have to deal with angry analysts.
The solution
A tool that automatically scans your power bi workspace, normalizes the response and then pushes it into unity catalog!
Extract semantic models from Power-BI and BYOL them in your databricks unity catalog. - GitHub - JeffDenzel/lineage-syncer: Extract semantic models from Power-BI and BYOL them in your databricks unity catalog.
