Hi @Julie285720, The merge operation in Databricks upserts data from a source table into a target table.
In your case, the source table people10mupdates and the target table is people10m.
The merge operation works as follows:
- It matches each record in the source table with the target table based on the condition provided in the ON clause. In your case, it's matching records where people10m.id = people10mupdates.id.
- If a match is found (WHEN MATCHED), it updates the existing record in the target table with the corresponding record from the source table.
- If no match is found (WHEN NOT MATCHED), it inserts the record from the source table into the target table. If your people10mupdates table became empty after the merge operation, it might be because all records were matched, updated, or inserted into the people10m table.
However, the merge operation does not delete records from the source table (people10mupdates in your case). If the people10mupdates table is empty, it might be due to another operation or process that deleted or moved the records.
Please check if any other operations or processes might be affecting the people10mupdates table.
