cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Merge didn't fail while inserting wrong data type values

elgeo
Valued Contributor II

Hello. During some example cases we were running, in order to identify how Databricks treats possible wrong actions we could make, we noticed that merge doesn't fail while inserting different data type values from the ones in the corresponding table. Please find the example below:

Target table schema

Target table schemaView used to update target table

Here you can see that we accidentally inserted the Status values in the Eff_Dt column (Date data type)

merge_failureMerge along with the success result

imageTarget table after the result

merge_failure3You can see that rows 10 & 11 inserted in the table without failing but with null value in EFF_DT (date type while we tried to insert string values). We would expect the merge to fail and not complete any of the DML actions. Moreover, we noticed that in this case in the identity column, id #9 was skipped.

Is this behavior expected? Could you please provide any insight? Thank you in advance

1 REPLY 1

elgeo
Valued Contributor II

Hello. Any update on this please? Thank you in advance

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group