Cannot resolve due to data type mismatch: incompatible types ("STRING" and ARRAY<STRING>
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-22-2024 07:01 AM
hey i have this error from a while :
Cannot resolve "(needed_skill_id = needed_skill_id)" due to data type mismatch: the left and right operands of the binary operator have incompatible types ("STRING" and "ARRAY<STRING>"). SQLSTATE: 42K09;
and these are the lines responsible for that :
to_update = current_link_df.where(col("active") == True).join(dataframe, ["position_id", "needed_skill_id"], how="left_anti")
(
DeltaTable.forName(spark, table_name).alias("t")
.merge(to_update.withColumn("end_date", current_date()).alias("s"), "s.position_id = t.position_id AND s.needed_skill_id = t.needed_skill_id")
Any idea ?
Thank you !!
