Hi all,
I have a question which is likely very fundamental. I am passing data from hive to snowflake using pyspark. My query is very simple - "select from table limit 100". The table I am querying is very large, but this query can be shown to the console via .show() in under 1 second, yet it takes 6 hours to pass the same tiny data set to snowflake.
Again, I understand this is likely very fundamental, just wondering the most likely causes of the issue.
