Thanks for response.

1. The target table is around 14GB, based on best practice from databricks, if table is less than 1TB we don't add partitions. 

2.Both source and target are delta tables. We enabled  spark.databricks.delta.optimizeWrite.enabled as true, so the target table file size is around 128M.Source table is 102GB in around 250million rows.  But the source in MERGE is a delta incremental reading from a source table. The reading is based on checkpoint so I think likely source table reading doesn't have issue (but source table does exist small file issues from 10kb to dozens of mb).

3. We want to know what MERGE needs to do behind the scene to lead a high driver memory usage.