https://community.databricks.com/t5/data-engineering/query-pushdown-in-snowflake/m-p/15530#M9846 <P>Hi,</P><P></P><P>I am wondering what documentation exists on Query Pushdown in Snowflake.</P><P>I noticed that a single function (monitonically_increasing_id()) prevented the entire query being pushed down to Snowflake during an ETL process. </P><P></P><P>Is Pushdown coming from the Spark JDBC connector OR is this from Databricks itself?</P><P></P><P>Is it possible to determine whether the query will be pushed down?</P> Mon, 13 Sep 2021 23:24:14 GMT Sam 2021-09-13T23:24:14Z https://community.databricks.com/t5/data-engineering/query-pushdown-in-snowflake/m-p/15530#M9846 <P>Hi,</P><P></P><P>I am wondering what documentation exists on Query Pushdown in Snowflake.</P><P>I noticed that a single function (monitonically_increasing_id()) prevented the entire query being pushed down to Snowflake during an ETL process. </P><P></P><P>Is Pushdown coming from the Spark JDBC connector OR is this from Databricks itself?</P><P></P><P>Is it possible to determine whether the query will be pushed down?</P> Mon, 13 Sep 2021 23:24:14 GMT https://community.databricks.com/t5/data-engineering/query-pushdown-in-snowflake/m-p/15530#M9846 Sam 2021-09-13T23:24:14Z https://community.databricks.com/t5/data-engineering/query-pushdown-in-snowflake/m-p/15532#M9848 <P>Hi Sam,</P><P></P><P>The Spark Connector applies predicate and query pushdown by capturing and analyzing the Spark logical plans for SQL operations. When the data source is Snowflake, the operations are translated into a SQL query and then executed in Snowflake to improve performance.</P><P></P><P>However, because this translation requires almost a one-to-one translation of Spark SQL operators to Snowflake expressions, not all of Spark SQL operators can be pushed down. When pushdown fails, the connector falls back to a less-optimized execution plan. The unsupported operations are instead performed in Spark.</P><P></P><P><A href="https://docs.snowflake.com/en/user-guide/spark-connector-use.html#pushdown" alt="https://docs.snowflake.com/en/user-guide/spark-connector-use.html#pushdown" target="_blank">https://docs.snowflake.com/en/user-guide/spark-connector-use.html#pushdown</A></P> Mon, 11 Oct 2021 16:18:18 GMT https://community.databricks.com/t5/data-engineering/query-pushdown-in-snowflake/m-p/15532#M9848 siddhathPanchal 2021-10-11T16:18:18Z