- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-29-2025 07:25 PM
Hey Hung, this is a pretty common issue when working with Delta Live Tables (DLT) and schema evolution. When the data type of a column changes—like in your case from Integer to Float—Delta sees that as an incompatible schema change, and by default, DLT doesn’t allow automatic merging of those types to avoid data quality issues. The cleanest solution is to explicitly cast the column in your DLT transformation code so the output matches the existing table schema. For example, you can cast reference_score to FLOAT using something like col("reference_score").cast("float"), which helps DLT see the schema as compatible. This avoids needing a full refresh. If casting doesn’t fit your case or if you're okay with refreshing the data, then you can also trigger a full refresh by deleting the target table and letting DLT recreate it with the new schema. But in most cases, a simple cast in your transformation logic is enough to resolve this smoothly. Let me know if you’d like help updating your DLT code!
Regards,
Brahma