Wojciech_BUK
Valued Contributor III

I think you can explore DLT API "Apply Changes". 
You can run it only in DLT pipeline but it can read from streming endpoint or streaming table.
Please check docs:
https://docs.databricks.com/en/delta-live-tables/cdc.html#language-python
You just include this line of code:

 

  stored_as_scd_type = "2"

 

and you have your SCD2 logic done 🙂 

But if you want to do clasic engineering you are right, this is MERGE with 

 

whenMatchedUpdate

 

You can do it either in pysaprk or spraksql and you can do it also in delta table straming.

Please let me know if that helped or maybe I missunderstood your question.