When running a Merge, if records from the table are deleted are the underlying files that contain the records deleted as well?

[User16826992666]

I know I have the option to delete rows from a Delta table when running a merge. But I'm confused about how that would actually affect the files that contain the deleted records. Are those files deleted, or are they rewritten, or what?

[sajith_appukutt · Accepted solution]

Delta implements MERGE by physically rewriting existing files.

It is implemented  in two steps.

1. Perform an inner join between the target table and source table to select all files that have matches.
2. Perform an outer join between the selected files in the target and source tables and write out the updated/deleted/inserted data.

Here is an article that explain the DML internals of delta - https://databricks.com/blog/2020/09/29/diving-into-delta-lake-dml-internals-update-delete-merge.html

The old files that are not referenced anymore would get garbage collected eventually when you run VACUUM