- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-11-2026 10:03 AM
Thank you both for replying. We are indeed using the natural keys from source for auto-cdc keys. The Identity column is not used in auto-cdc but needs to be populated as an added column in my silver tables. Users need that to join multiple tables in gold layer. Now the problem I am facing is with auto-cdc I am not able to populate that directly in silver as there is no such option.
However, I found an alternative way where we create the tables upfront in a separate script. Looks like I cannot do this in python as dp.create_streaming_table() didn't support in my case (may be there is some way here too), but I am able to achieve it using a .sql script. The .sql script needs to be part of the declarative pipeline though.
Example:
The issue I have now is I am not aware of any potential risk of doing this yet. Any insight would be helpful.