cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

DSam05
by New Contributor
  • 1658 Views
  • 3 replies
  • 1 kudos

10.4LTS has outdated snowflake spark connector, how to force latest snowflake spark connector

Hi,I am trying to run my code from a scala fatjar on azure-databricks, which connects to snowflake for the data.I usually run my jar on 9.1 LTS.However when I run on 10.4 LTS the performace was 4x degraded and in the log it says WARN SnowflakeConnect...

  • 1658 Views
  • 3 replies
  • 1 kudos
Latest Reply
slavat
New Contributor II
  • 1 kudos

I also encountered the similar problem. This is a snippet from my log file:22/12/18 09:36:28 WARN SnowflakeConnectorUtils$: Query pushdown is not supported because you are using Spark 3.2.0 with a connector designed to support Spark 3.1. Either use t...

  • 1 kudos
2 More Replies
Labels