Hmm, CDC is not built for full loads, so don't think that is the best option here.I can clearly see the logic behind you're setup, but also see there's a possibility for optimization and cleanliness, as you're doing a lot of comparisons just to figur...
Hello,It's fairly common to not have the SCD col/tag in bronze layer and adding it in silver, as the granularity in bronze (often) don't have collisions on unique ID's and it is rarely something you care about in bronze. Then in silver you either add...
Hey Phani!Cool setup you have there - some comments and ideas:Generally it sounds like you have a good apporch - Setting up a dedicated dev environement apart for staging and prod is the way. However, restricting access to tables in dev is generally ...
there could be several different reasons, but mainly, it's because grouping arbitrary data into some target file-size is well... arbitrary.Imagine I gave you a large container of sand and some emtpy buckets, and asked you to move the sand from the co...
Is your dbfs mounted?Otherwise, try uploading it to your workspace's "shared" folder - this is a common place to put these sorts of files. dbfs is slowly getting phased out and not really in any best practices.
