NandiniN
Databricks Employee
Databricks Employee

Hi @yvishal519 

  • Avoid Circular Dependency: The employee_stg view should not directly reference the silver_employee table. Instead, the apply_changes function will handle the merging logic.
  • Use LIVE Keyword: Ensure that the source table in apply_changes is referenced with the LIVE keyword.
  • Audit Columns: The __CreatedDateTime and __UpdatedDateTime columns are managed within the apply_changes function to ensure correct handling of timestamps.

This approach should help you avoid the circular dependency issue and correctly manage the audit columns in your DLT pipeline.

Docs - https://docs.databricks.com/en/delta-live-tables/cdc.html

Thanks!