- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-26-2023 09:04 PM
if I have two dataframes df_target and df_source, can I do df_target.as("t).merge(df_source.as("s"), "s.id=t.id").whenMatched().updateAll().whenNotMatched.insertAll.execute().
when I tried the code above, I got the error "merge is not a member of the dataframe". If that is the case, what is the best way to do it? Thanks!
Accepted Solutions
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-27-2023 03:43 AM
Hi @andrew li Merge functionality is a part of Delta Lake API and not native dataframe API. Only when you are writing the data in delta format, you can use this functionality.
Please check the below link to achieve Merge using delta lake API.
https://learn.microsoft.com/en-us/azure/databricks/delta/merge
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-27-2023 03:43 AM
Hi @andrew li Merge functionality is a part of Delta Lake API and not native dataframe API. Only when you are writing the data in delta format, you can use this functionality.
Please check the below link to achieve Merge using delta lake API.
https://learn.microsoft.com/en-us/azure/databricks/delta/merge
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-27-2023 04:58 AM
nice info
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-28-2023 01:22 AM
@Aviral Bhardwaj Please upvote 🙂
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-27-2023 09:10 PM
Hi @andrew li
Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help.
We'd love to hear from you.
Thanks!
![](/skins/images/97567C72181EBE789E1F0FD869E4C89B/responsive_peak/images/icon_anonymous_message.png)
![](/skins/images/97567C72181EBE789E1F0FD869E4C89B/responsive_peak/images/icon_anonymous_message.png)