Can't mergeSchema handle int and bigint?

Dhruv-22
Contributor III

‎10-21-2025 04:25 AM

I have a table which has a column of data type 'bigint'. While overwriting it with new data, given that I do full loads, I used 'mergeSchema' to handle schema changes. The new data's datatype was int. I thought mergeSchema can easily handle that, but it throws an error.

crm_retail_df.write.mode("overwrite").format("delta").option('mergeSchema', True).saveAsTable(silver_table)

-- Output
[DELTA_FAILED_TO_MERGE_FIELDS] Failed to merge fields 'Lob_Pk' and 'Lob_Pk' SQLSTATE: 22005

 Shouldn't 'mergeSchema' automatically widen the incoming from int to bigint? If not, then does mergeSchema only handle cases when datatypes match between new and old data?

0 Kudos