- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-15-2024 08:57 AM
Hello community,
I'm inserting in a table defined (with primary key and foreign key set) some records in overwrite mode every moment I run a workflow where the task is defined. Why after inserting those records the DDL schema changes? Why I have my primary and foreign keys unset? Seems that changes the schema!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-15-2024 03:43 PM
Hi @jeremy98,
When you use the "insert overwrite" mode in Databricks, it can lead to the schema being reset, which includes the removal of primary and foreign key constraints. This happens because the "insert overwrite" operation essentially replaces the entire table, which can result in the loss of any constraints that were previously defined
To maintain your primary and foreign key constraints, you might consider using a different strategy for updating your table. For example, using the "merge" strategy instead of "insert overwrite" can help preserve the schema and constraints. The "merge" strategy allows you to update, insert, or delete records based on a condition, which can be more efficient and less disruptive to the schema