Delta table upsert - databricks community

weldermartins
Honored Contributor

Hello guys,

I'm trying to use upsert via delta lake following the documentation, but the command doesn't update or insert newlines.

scenario: my source table is separated in bronze layer and updates or inserts are in silver layer.

from delta.tables import *
 
deltaTableVendas = DeltaTable.forPath(spark, '/user/hive/warehouse/bronze.db/vendas')
deltaTableVendasUpdates = DeltaTable.forPath(spark, '/user/hive/warehouse/silver.db/vendas')                                                   
 
dfUpdates = deltaTableVendasUpdates.toDF()
 
deltaTableVendas.alias('vendas') \
  .merge(
    dfUpdates.alias('updates'),
    'vendas.numero_transacao = updates.numero_transacao'
  ) \
  .whenMatchedUpdate(set =
    {
      "numero_transacao": "updates.numero_transacao",
      "numped": "updates.numped",
      "codcli": "updates.codcli",
      "codprod": "updates.codprod",
      "data_venda": "updates.data_venda",
      "quantidade": "updates.quantidade",
      "valor": "updates.valor"
      
    }
  ) \
  .whenNotMatchedInsert(values =
    {
      "numero_transacao": "updates.numero_transacao",
      "numped": "updates.numped",
      "codcli": "updates.codcli",
      "codprod": "updates.codprod",
      "data_venda": "updates.data_venda",
      "quantidade": "updates.quantidade",
      "valor": "updates.valor"
    }
  ) \
  .execute()