DLT-META requires ddl when using cdc_apply_changes
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-16-2024 03:27 PM
We are setting up new DLT Pipelines using the DLT-Meta package. Everything is going well in bringing our data in from Landing to our Bronze layer when we keep the onboarding JSON fairly vanilla. However, we are hitting an issue when using the cdc_apply_changes feature. As soon as we add that feature, the package will look for a .ddl schema file. Since we aren't using one, the package will fail when we start the pipelines. Complaining it can't parse a null file
For a bit of background, we plan on using schema evolution and would like to use the cdc_apply_changes to keep the bronze tables mostly similar to the source systems.
The cdc_apply_changes doesn't need a schema file, so I am confused about why one is now required. Is this just a bug in the DLT-Meta package? Does anyone have any suggestions?
- Labels:
-
Workflows