what are the join hints, available in spark 3.0, and how does it help compare to pervious spark version

[User16826994223]

what are the join hints, available in spark 3.0, and how does it help compare to pervious spark version 

[Srikanth_Gupta_ · Accepted solution]

4 types of join hints in Spark 3.0

BROADCAST

MERGE

SHUFFLE_HASH

SHUFFLE_REPLICATE_NL

May be good idea to enable Adaptive Query Execution which speeds up Spark SQL join during run time

In Spark 3.0, Adaptive Query Execution comes with below features

Dynamically coalescing shuffle partitions

Dynamically switching join strategies

Dynamically optimizing skew joins

more details on ADE are available here