โ03-30-2022
04:55 AM
- last edited on
โ03-21-2025
06:00 AM
by
Advika
Hello. Databricks can be connected with PowerBi to visualize data. The process is described here. Of the various approaches described there, what is the fastest for large amounts of data? Are there even any difference in performance for the different approaches? Are there other approaches than the ones mentioned above for importing large amounts of data into BI tools?
Thanks.
โ05-16-2022 11:04 PM
Hi @sondrewbโ you can connect BI tools to Databricks SQL endpoints to query data in tables through an ODBC/JDBC protocol integrated in our Simba drivers. With Cloud Fetch, which we released in Databricks Runtime 8.3 and Simba ODBC 2.6.17 driver, we introduce a new mechanism for fetching data in parallel via cloud storage such as AWS S3 and Azure Data Lake Storage to bring the data faster to BI tools. In our experiments using Cloud Fetch, we observed a 10x speed-up in extract performance due to parallelism.
Please find the below document reference
โ03-30-2022 05:04 AM
SQL endpoint. Question How big is the dataset? As usually limitation is dataset size on the PowerBI as there is a limit of 1GB for pro and 10GB for premium. 10GB for Spark is a small dataset but large for PowerBI.
โ03-31-2022 04:58 AM
@Hubert Dudekโ , Thanks for the quick response. Could you point me to an overview/comparison of the performance of the different ways to connect? Why is SQL Endpoint faster?
โ05-16-2022 11:04 PM
Hi @sondrewbโ you can connect BI tools to Databricks SQL endpoints to query data in tables through an ODBC/JDBC protocol integrated in our Simba drivers. With Cloud Fetch, which we released in Databricks Runtime 8.3 and Simba ODBC 2.6.17 driver, we introduce a new mechanism for fetching data in parallel via cloud storage such as AWS S3 and Azure Data Lake Storage to bring the data faster to BI tools. In our experiments using Cloud Fetch, we observed a 10x speed-up in extract performance due to parallelism.
Please find the below document reference
โ06-07-2022 10:01 AM
Hi @Soner Candanโ,
Just a friendly follow-up. Do you still need help or our community responses helped? Please let us know.
Passionate about hosting events and connecting people? Help us grow a vibrant local communityโsign up today to get started!
Sign Up Now