topic Re: merge vs MERGE INTO in Data Engineering

merge vs MERGE INTO

ros — Wed, 31 May 2023 07:47:59 GMT

from 10.4 LTS version we have low shuffle merge, so merge is more faster. But what about MERGE INTO function that we run in sql notebook of databricks.

Is there any performance difference when we use databrciks pyspark ".merge" function vs databricks sql "MERGE INTO" function. And does both have "low shuffle merge"

Re: merge vs MERGE INTO

Ajay-Pandey — Wed, 31 May 2023 12:25:32 GMT

Hi @Roshan RC ,

There is no difference between both as the internal physical plan will be the same for both of the codes.

#DAIS2023

Re: merge vs MERGE INTO

Anonymous — Thu, 01 Jun 2023 07:10:35 GMT

Hi @Roshan RC

Thank you for posting your question in our community! We are happy to assist you.

To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?

This will also help other community members who may have similar questions in the future. Thank you for your participation and let us know if you need any further assistance!