cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
Erik
by • Valued Contributor III
  • 20887 Views
  • 19 replies
  • 15 kudos

How to enable/verify cloud fetch from PowerBI

I tried to benchmark the Powerbi Databricks connector vs the powerbi Delta Lake reader on a dataset of 2.15million rows. I found that the delta lake reader used 20 seconds, while importing through the SQL compute endpoint took ~75 seconds. When I loo...

query_statistics query_profile_tree_view
Data Engineering
Reply
  • 20887 Views
  • 19 replies
  • 15 kudos
View Replies
Latest Reply
datadrivenangel
New Contributor II
  • 15 kudos

I'm troubleshooting slow speeds (~6Mbps) from Azure Databricks to the PowerBI Service (Fabric) via dataflows.Drivers are up to date. PowerBI is using Microsoft's Spark ODBC driver Version 2.7.6.1014, confirmed via log4j.HybridCloudStoreResultHandler...

  • 15 kudos
18 More Replies
DianGermishuiz1
by • New Contributor III
  • 8042 Views
  • 7 replies
  • 4 kudos

Resolved! Unsupported cloud file system schema 'wasbs' error in custom catalog, but not in hive_metastore

When trying to run a create table USING CSV that pulls data from an azure blob storage path in a custom catalog I created, I get an error stating Unsupported cloud file system schema 'wasbs'.However, when I run this code in the hive_metastore catalog...

wasbs cloud path unsupported error
Data Engineering
Reply
  • 8042 Views
  • 7 replies
  • 4 kudos
View Replies
Latest Reply
youssefmrini
Databricks Employee
  • 4 kudos

WASB Has been deprecated.

  • 4 kudos
6 More Replies
data_testing1
by • New Contributor III
  • 74153 Views
  • 6 replies
  • 13 kudos

Can databricks be used locally to learn it or is it cloud only

I'm tired of telling clients or referrals I don't know databricks but it seems like the only option is to have a big AWS account and then use databricks on that data. Can I download it locally for training, upskilling with python or is it only for cl...

Data Engineering
Reply
  • 74153 Views
  • 6 replies
  • 13 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 13 kudos

Thanks for linking directly to the docker image @Hubert Dudek​ ! And thanks for the info @Prabakar Ammeappin​ and @Amit Nainawati​ @Andrew Schell​ Let us know if you have more questions! If not, choose a best answer in this thread and let us know how...

  • 13 kudos
5 More Replies
edwardh
by • New Contributor III
  • 4680 Views
  • 5 replies
  • 6 kudos

How to call Cloud Fetch APIs?

About Cloud Fetch mentioned in this article:https://databricks.com/blog/2021/08/11/how-we-achieved-high-bandwidth-connectivity-with-bi-tools.htmlAre there any public APIs that can be called directly without ODBC or JDBC drivers? Thanks.

Data Engineering
Reply
  • 4680 Views
  • 5 replies
  • 6 kudos
View Replies
Latest Reply
edwardh
New Contributor III
  • 6 kudos

Hi @Kaniz Fatma​, can you please give some help on this question? Thanks

  • 6 kudos
4 More Replies
aldrich_ang
by • New Contributor II
  • 8706 Views
  • 3 replies
  • 4 kudos

Resolved! How do you use cloud fetch?

We're trying to pull a big amount of data using databricks sql and seem to have a bottleneck on network throughput when fetching the data.I see there's a new feature called cloud fetch and this seems to be the perfect solution for our issue. But I do...

Data Engineering
Reply
  • 8706 Views
  • 3 replies
  • 4 kudos
View Replies
Latest Reply
-werners-
Esteemed Contributor III
  • 4 kudos

Trying to get an idea of what you are trying:so you query directly on a database of +100GB or is it parquet/delta source?Also, where is the result fetched to? File download, BI tool, ...?

  • 4 kudos
2 More Replies
User16826994223
by • Honored Contributor III
  • 1641 Views
  • 2 replies
  • 0 kudos

What is the differentiator between delta sharing and other cloud sharing platform

What is differentiator between delta sharing and other cloud sharing platform.

Data Engineering
Reply
  • 1641 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
User16752244127
Contributor
  • 0 kudos

Also, unlike other servers, Delta Sharing internally uses pre-signed URLs to S3, GCS, or ADSL, so data transfer from a client happens at the bandwidth of the underlying cloud object-store. This way the Delta Sharing server scales extremely well and d...

  • 0 kudos
1 More Replies
Labels