If I call powerbi refresh from databricks workflow job it waits until the refresh is complete unlike ADF(Azure Data Factory) pbi refresh through rest api. In case of big pbi dataset which might take more than a hour to refresh in databricks compute remains on for the whole timebeing as it waits until pbi refresh is complete.

Whereas in ADF it is a asynchronous rest api call which just hit refesh and doesn't wait.

What could be optimal solution to avoid unnecessary wait in databricks workflow job.

#powerbi #Databricksjob