Hi all,
โ
I have a question which is likely very fundamental. I am passing data from hive to snowflake using pyspark. My query is very simple - "select from table limit 100".โ The table I am querying is very large, but this query can be shown to the console via .show() in under 1 second, yet it takes 6 hours to pass the same tiny data set to snowflake.
โ
Again, I understand this is likely very fundamental, just wondering the most likely causes of the issue.โ
