https://community.databricks.com/t5/data-engineering/simple-pyspark-query-very-slow-in-pushing-to-snowflake/m-p/22391#M15326 <P>Hi all,</P><P>​</P><P>I have a question which is likely very fundamental. I am passing data from hive to snowflake using pyspark. My query is very simple - "select from table limit 100".​ The table I am querying is very large, but this query can be shown to the console via .show() in under 1 second, yet it takes 6 hours to pass the same tiny data set to snowflake.</P><P>​</P><P>Again, I understand this is likely very fundamental, just wondering the most likely causes of the issue.​</P> Tue, 15 Nov 2022 01:58:05 GMT csw77 2022-11-15T01:58:05Z https://community.databricks.com/t5/data-engineering/simple-pyspark-query-very-slow-in-pushing-to-snowflake/m-p/22391#M15326 <P>Hi all,</P><P>​</P><P>I have a question which is likely very fundamental. I am passing data from hive to snowflake using pyspark. My query is very simple - "select from table limit 100".​ The table I am querying is very large, but this query can be shown to the console via .show() in under 1 second, yet it takes 6 hours to pass the same tiny data set to snowflake.</P><P>​</P><P>Again, I understand this is likely very fundamental, just wondering the most likely causes of the issue.​</P> Tue, 15 Nov 2022 01:58:05 GMT https://community.databricks.com/t5/data-engineering/simple-pyspark-query-very-slow-in-pushing-to-snowflake/m-p/22391#M15326 csw77 2022-11-15T01:58:05Z