- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-22-2025 08:26 AM
How are you doing today? As per my understanding, The error happens because _change_type is appearing twice in your schema, likely due to how you're processing the Change Data Feed (CDF). This can happen if SELECT * is pulling in multiple copies of _change_type or if the global temp view is duplicating columns. To fix this, try explicitly selecting only the necessary columns in your query_cdf, check df_upsert.printSchema() to confirm duplicates, and make sure _change_type isn’t unintentionally added again in the merge conditions. Also, avoid using createOrReplaceGlobalTempView() if it's causing schema issues. Give these a shot and see if it helps!
Regards,
Brahma