- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-08-2025 02:48 PM
Hi @excavator-matt ,
Yes, you are correct. CloudFiles/Autoloader handles idempotency on the file level.
From the guide's perspective, the View is created from the source files in the specified location. This view captures all files and their corresponding rows, and then AutoCDC is applied from the view to create the baseline streaming table.
Regarding schema evolution and enforcement, IMO, it is about the right balance: In Bronze, your data changes, so we need to be resilient enough to handle them. However, as we move through the Silver and Gold layers, we may require a more rigid schema and enforce it to ensure that downstream/consumer systems don't break.
The key detail is that AUTO CDC handles schema evolution. Note that the streaming table “rdbms_orders” is created without a schema, and then AUTO CDC FLOW is applied from the “full_orders_snapshot” view.
I hope this helps. Thank you!