filipniziol
Esteemed Contributor

Hi @MikeGo ,

In general the MERGE statement can be memory-intensive.

To optimize check this article:
https://medium.com/@abhinav.prakash1804/databricks-a-comprehensive-optimization-guide-7b803289c5a7

If you need an additional help, could you write:
1. Are you partitioning/clustering the data by the columns used in JOIN? If yes, how many partitions do you have?

2. What is the average file size of source and target (run DESCRIBE DETAILS on source and target tables)?

3. When doing the merge to the target of 220 million, how many records are in the source?