pichlerpa
New Contributor III

Hi everyone, we are facing exactly the same problem, result fetching takes far too long by connecting remotely via R Studio and ODBC for interactive workloads. We made sure to use the latest version of the ODBC Databicks connector together with the latest Databricks runtime with Cloud Fetch enabled. Unfortunately, without any effect. We tried:

  • scaling up the cluster
  • tweaking the connector (RowsFetchedPerBlock, UseNativeQuery, etc.)
  • using Databricks Connect with Apache Arrow as an alternative (untraceable OOM error by transferring (collect) Spark DataFrame to local R)

Databricks Connect OOM