How to remove duplicates in a Delta table?
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-19-2023 01:48 AM
I made multiple inserts (by error) in a Delta table and I have now strict duplicates, I feel like it's impossible to delete them if you don't have a column "IDENTITY" to distinguish lines (the primary key is RLOC+LOAD_DATE):
it sounds odd to me not to be able to do any anything after data definition for removing strict identical rows. I wonder if there was still a way to distinguish them by physical address or insertion date?
https://www.sqlshack.com/different-ways-to-sql-delete-duplicate-rows-from-a-sql-table/
Otherwise I will do that: