Databricks

Raghu_Bindingan · 04-12-2023

Hi I am trying to use the SQL Merge statement on databricksMERGE INTO targetUSING sourceON source.key = target.keyWHEN MATCHED UPDATE SET *WHEN NOT MATCHED INSERT *WHEN NOT MATCHED BY SOURCE DELETEThis is failing with the error [PARSE_SYNTAX_ERROR...

Raghu_Bindingan · 12-03-2022

Has anyone attempted to truncate a delta live gold level table that gets populated via a pipeline and then tried to repopulate it by starting the pipeline. I have this situation wherein i need to reprocess all data in my gold table, so i stopped the ...

Raghu_Bindingan · 04-13-2023

I was missing the THEN before UPDATE, INSERT and DELETE. This keyword is missing from the documentation on Databricks https://learn.microsoft.com/en-us/azure/databricks/delta/mergeIt now works. Thanks

Raghu_Bindingan · 12-03-2022

Tried that too but it is failing to repopulate the data, looks like some metadata is being stored somewhere which is causing this

Raghu_Bindingan · 12-03-2022

Thanks @Hubert Dudek I will try dropping and creating the table.

Raghu_Bindingan · 12-02-2022

HI @Johan Van Noten In the instance that i had which was quite simple it did perform ok, but you are right, about performance, this is something that needs to be monitored. I am sure SQL server would go crazy with this approach.ThanksRaghu

Raghu_Bindingan · 12-02-2022

Hi @Johan Van Noten I got it to work, its a very different kind of SQL then what i am used to but it works, you could try the below and tweak it to your needs. Basically spark sql does not like predicates that are not equal, so you will need to use ...

Databricks

User Stats

User Activity

SQL Merge Statement not working

Truncate delta live table and try to repopulate it in the pipeline

Re: SQL Merge Statement not working

Re: Truncate delta live table and try to repopulate it in the pipeline

Re: Truncate delta live table and try to repopulate it in the pipeline

Re: Correlated column exception in SQL UDF when using UDF parameters.

Re: Correlated column exception in SQL UDF when using UDF parameters.