cancel
Showing results for 
Search instead for 
Did you mean: 
Warehousing & Analytics
cancel
Showing results for 
Search instead for 
Did you mean: 

Fastest way to get data into PowerBI

sondrewb
New Contributor III

Hello. Databricks can be connected with PowerBi to visualize data. The process is described here. Of the various approaches described there, what is the fastest for large amounts of data? Are there even any difference in performance for the different approaches? Are there other approaches than the ones mentioned above for importing large amounts of data into BI tools?

Thanks.

1 ACCEPTED SOLUTION

Accepted Solutions

Anonymous
Not applicable

Hi @sondrewb​ you can connect BI tools to Databricks SQL endpoints to query data in tables through an ODBC/JDBC protocol integrated in our Simba drivers. With Cloud Fetch, which we released in Databricks Runtime 8.3 and Simba ODBC 2.6.17 driver, we introduce a new mechanism for fetching data in parallel via cloud storage such as AWS S3 and Azure Data Lake Storage to bring the data faster to BI tools. In our experiments using Cloud Fetch, we observed a 10x speed-up in extract performance due to parallelism.

Please find the below document reference

https://databricks.com/blog/2021/08/11/how-we-achieved-high-bandwidth-connectivity-with-bi-tools.htm...

View solution in original post

4 REPLIES 4

Hubert-Dudek
Esteemed Contributor III

SQL endpoint. Question How big is the dataset? As usually limitation is dataset size on the PowerBI as there is a limit of 1GB for pro and 10GB for premium. 10GB for Spark is a small dataset but large for PowerBI.

@Hubert Dudek​ , Thanks for the quick response. Could you point me to an overview/comparison of the performance of the different ways to connect? Why is SQL Endpoint faster?

Anonymous
Not applicable

Hi @sondrewb​ you can connect BI tools to Databricks SQL endpoints to query data in tables through an ODBC/JDBC protocol integrated in our Simba drivers. With Cloud Fetch, which we released in Databricks Runtime 8.3 and Simba ODBC 2.6.17 driver, we introduce a new mechanism for fetching data in parallel via cloud storage such as AWS S3 and Azure Data Lake Storage to bring the data faster to BI tools. In our experiments using Cloud Fetch, we observed a 10x speed-up in extract performance due to parallelism.

Please find the below document reference

https://databricks.com/blog/2021/08/11/how-we-achieved-high-bandwidth-connectivity-with-bi-tools.htm...

Hi @Soner Candan​,

Just a friendly follow-up. Do you still need help or our community responses helped? Please let us know.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.