the drop() only removes the specific data frame instance of the column. So if you have:

val new_ddf = ddf.join(up_ddf, "name")

then in new_ddf you have two columns ddf.name and up_ddf.name.

val new_ddf = ddf.join(up_ddf, "name").drop(up_ddf.col("name") will remove that column and only leave ddf.name in new_ddf.