In DBR 17 (Spark 4.0), spark.sql.ansi.enabled defaults to true, so implicit coercions (like comparing INT to STRING) will parse the string and raise a runtime error if it’s malformed; to keep pipelines resilient, avoid relying on implicit casts and instead use safe casts such as try_cast or the ?:: operator to return NULL on bad input, or pre-filter/normalize strings before casting; note that :: is a strict cast alias of CAST and will error under ANSI, so use ?::/try_cast for NULL-on-error semantics, and only consider disabling ANSI (SET ANSI_MODE=false or SET spark.sql.ansi.enabled=false) if you explicitly want legacy leniency at the cost of data-quality protections.