Power BI Dataflow Gen1 → Databricks Pipeline: Is There a Migration Accelerator or Converter?

[nafikazi]

Hello Great Analytics professionals:

Many organizations have legacy Power BI Dataflow Gen1 solutions and are now considering moving their ETL workloads to Databricks.

Is there an existing Databricks migration accelerator, converter, or framework that can take the logic from Dataflow Gen1:particularly Power Query (M), SQL queries, transformations, joins, and CTEs, and convert or translate it into Databricks pipelines (SQL/PySpark)?

I'm especially interested in tools or approaches that can help with a proof of concept (POC) to demonstrate to leadership:

• How much of the existing ETL logic can be migrated or automated
• Migration effort and complexity
• Databricks performance, scalability, and maintainability benefits

Has anyone implemented this type of Dataflow Gen1 → Databricks migration at enterprise scale? Would appreciate any accelerators, frameworks, GitHub projects, or lessons learned.
appreciate your advice.

Thanks
#migration #innovation #unitycatalog #pipeline

[andrew635james]

There is no native tool to automatically convert proprietary Power Query (M) code into Databricks PySpark or SQL. For your POC, you can use automated translation tools like SQLGlot for embedded SQL and CTEs, but complex M transformations will require manual refactoring into PySpark or dbt, allowing you to focus the pitch on Databricks' scalability, Unity Catalog governance, and performance benefits.

[nafikazi]

Thank you james.

[tran073huyen]

Hello, There isn’t a direct one-click Dataflow Gen1 → Databricks converter, particularly for Power Query M. Databricks has migration accelerators for SQL/ETL workloads, but M transformations will generally need to be translated and validated separately. For a POC, I’d take a representative set of Dataflows, categorize the M/SQL transformations, then measure what can be automated versus what needs manual refactoring. 

[nafikazi]

Thank you

[data_pulse · Accepted solution]

@nafikazi 

There isn't a one-click Power BI Dataflow Gen1 / Power Query M to Lakeflow conversion path yet available.

For a POC, decompose each flow and use Genie Code to accelerate the translation into Databricks SQL/ PySpark.

Provide the M query, source/target schema and expected output, then use prompts like:

Explain this Power Query M script step-by-step and identify
the filters, joins, derived columns and aggregations.

Convert this M query to Databricks SQL.
Preserve the transformation logic and flag anything
that has no direct equivalent.

Convert this M query to PySpark for a Lakeflow pipeline.
Highlight any Power Query-specific functions that need manual redesign.

Then validate the generated output against the existing Dataflow using row counts and a few business-rule checks.

For a larger migration, first classify flows into straightforward SQL transformations vs Power Query specific/custom logic. Automate the simpler ones and handle complex cases seperately.

This could be one of the work arounds until there is a more plausible solution/connector to do this in Databricks in near future.

[nafikazi]

Very well explained. Appreciate your help. Thank you.