Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-31-2024 11:59 AM
Hi @yvishal519
- Avoid Circular Dependency: The
employee_stgview should not directly reference thesilver_employeetable. Instead, theapply_changesfunction will handle the merging logic. - Use
LIVEKeyword: Ensure that the source table inapply_changesis referenced with theLIVEkeyword. - Audit Columns: The
__CreatedDateTimeand__UpdatedDateTimecolumns are managed within theapply_changesfunction to ensure correct handling of timestamps.
This approach should help you avoid the circular dependency issue and correctly manage the audit columns in your DLT pipeline.
Docs - https://docs.databricks.com/en/delta-live-tables/cdc.html
Thanks!