12-02-2021 03:53 PM
Hello,
I've noticed that Collect_Set and Collect_List are not pushed down to the database?
Runtime DB 9.1LTS
Spark 3.1.2
Database: Snowflake
Is there any way to get a distinct set from a group by in a way that will push down the query to the database?
12-02-2021 11:29 PM
Hm so collect_set does not get translated to listagg.
Can you try the following?
never-displayed
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
Click here to register and join today!
Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.