- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-12-2023 06:17 AM
I have a statement like this with pyspark:
target_tbl.alias("target")\
.merge(stage_df.hint("broadcast").alias("source"), merge_join_expr)\
.whenMatchedUpdateAll()\
.whenNotMatchedInsertAll()\
.whenNotMatchedBySourceDelete()\
.execute()
It is not accepting the broadcast hint. I am getting the following:
Join hint ignored: This query has a join hint '(strategy=broadcast)' that is not associated with any join operator and will thus be ignored. Investigate the query to see if the hint is placed correctly.
In this video:
https://www.youtube.com/watch?v=o2k9PICWdx0&t=797s
it is said that this approach is working
- Labels:
-
BroadcastJoin
-
Pyspark