- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-08-2022 09:16 AM
I had been trying to upsert rows into a table in Azure Blob Storage (ADLS Gen 2) based on two partitions (sample code below).
insert overwrite table new_clicks_table partition(client_id, mm_date)
select
click_id
,user_id
,click_timestamp_gmt
,campaign_id
,site_url
,client_id
,mm_date
from
old_clicks_table
where
mm_date between '2022-12-01' and '2022-12-07'
and
client_id in (123456)
;However, using "INSERT OVERWRITE" deleted all the data that was previously in that table, not just updating the data based on the give partitions (ie: all data in the table was deleted for client ID 123456 before 2022-12-01 and I was just left with data from 2022-12-01 thru 2022-12-07).
What's the easiest way to upsert this data into my table for only the specified partitions in my query and retain the rest of my data?
Can this be done using a combination of "INSERT" and "OVERWRITE" or do I need to use "MERGE"? If the latter, can someone provide the best example of how to write this data using "MERGE"?