cchalc
Databricks Employee
Databricks Employee

You would need to set EnableQueryResultsDownload Flag to 0 (zero) which will disable cloud fetch.

cchalc
Databricks Employee
Databricks Employee

So why is ColumnarToRow required?

pichlerpa
New Contributor III

Hi everyone, check out my latest blog post to verify whether or not cloudfetch is actually used, maybe you also find some other optimizations there:

https://medium.com/creative-data/boosting-databricks-odbc-driver-be2cf08a7a4a?sk=bd814e0c3d6a9b32beb...

pulkitm
New Contributor III

Guys, is there any way to switch off CloudFetch and fall back to ArrowResultSet by default irrespective of size? using the latest version of Spark Simba ODBC driver?

datadrivenangel
New Contributor III

I'm troubleshooting slow speeds (~6Mbps) from Azure Databricks to the PowerBI Service (Fabric) via dataflows.

  • Drivers are up to date. PowerBI is using Microsoft's Spark ODBC driver Version 2.7.6.1014, confirmed via log4j.
  • HybridCloudStoreResultHandler is being used. Confirmed via log4j.
  • MapPartitionInternals is NOT using CloudStoreCollector. ⚠️ The Spark DAG ends with mapPartitionsInternal at HybridResultCollector.scala.

Does this mean that CloudFetch is not enabled here?

In the neo4j logs I also see CloudStoreBasedResultHandler receiving and responding to getNextCloudStoreBasedSet which I interpret as cloudFetch being enabled?