Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-21-2025 07:02 AM
Hi Dhruv,
Delta won't automatically upcast unless you explicitly handle it. Cast the column Lob_Pk to LongType (which maps to BIGINT in SQL/Delta). Try below snippet
from pyspark.sql.functions import col
from pyspark.sql.types import LongType
crm_retail_df = crm_retail_df.withColumn("Lob_Pk", col("Lob_Pk").cast(LongType()))
crm_retail_df.write \
.mode("overwrite") \
.format("delta") \
.option("mergeSchema", "true") \
.saveAsTable(silver_table)
G.Chiranjeevi