I have monthly files which comes in S3 bucket. I want to implement SCD type2 in snowflake.
I am ok to read the new files, clean it.
My question is about comparing what I have read from the files, with what is stored in the snowflake table already(millions of records) and capturing the change.
Main question:
How to join the snowflake table to the view created from the monthly file.
1)I can read the snowflake table in data bricks like select * and can create a view and do the processing.
I am sure this is now how it's done in production.
Thanks,
Any pointers on how to solve the problem?
