At the Lakehouse, we don't enforce Primary Keys, which is why the deduplication strategy is so important. One of my favourites is using transformWithStateInPandas. Of course, it only makes sense in certain scenarios. See all five major strategies on my blog #databricks
https://databrickster.medium.com/deduplicating-data-on-the-databricks-lakehouse-5-ways-36a80987c716
https://www.sunnydata.ai/blog/databricks-deduplication-strategies-lakehouse
![Community [Industry BrickTalk #4] Transforming Commercial Real Estate Portfolio Management with AI](/t5/image/serverpage/image-id/25529iFB6AFAFAE433451D/image-size/large?v=v2&px=999)