This is a known issue with the Power BI/Databricks integration preview for refreshing Power BI datasets from Databricks workflows. Even when a job completes and the refresh history shows success, actual data changesโsuch as newly inserted rowsโare often not visible in the Power BI dataset. As you've observed, querying the dataset's XMLA endpoint will confirm that both the โlast processedโ timestamps and row counts remain unchanged.โ
Key facts on current behavior
-
The current preview can mark refreshes as "completed" without capturing your latest data changes to the dataset.โ
-
Setting refresh_after_update to true in "import" mode does not guarantee the dataset will update to reflect new or changed records.โ
-
This appears to be a limitation (or bug) in the preview, not intended final behavior. Several community posts confirm others are encountering the exact scenario you describe, and it is being discussed as an unresolved issue.โ
Recommended next steps
-
Monitor the official Databricks and Power BI release notes and forums for updates regarding this featureโs preview status.
-
If this block is impacting production, consider alternate dataset refresh mechanisms such as triggering Power BI REST API refreshes via Power Automate or Azure Data Factory, which are more mature and stable for now.โ
-
Report your findings to Microsoft/Databricks support or community forums, as additional user feedback is driving awareness of this preview issue.โ
In short, the data not being applied to the Power BI datasetโeven though the workflow signals a successful refreshโis a known preview limitation, and not the expected behavior for the final release.โ
