Is it possible to use merge command when source file is parquet and destination file is delta? Or both files must delta files?
Currently, I'm using this code and I transform parquet into delta and it works. But I want to avoid of this tranformation.
Thanks
from delta.tables import *
deltaTablePeople = DeltaTable.forPath(spark, 'abfss://destination-delta')
deltaTablePeopleUpdates = DeltaTable.forPath(spark, 'abfss://source-parquet')
dfUpdates = deltaTablePeopleUpdates.toDF()
deltaTablePeople.alias('people') \
.merge(
dfUpdates.alias('updates'),
'people.id = updates.id'
) \
.whenMatchedUpdate(set =...