Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-05-2026 02:38 AM
Hi I believe the spark configuration you are trying to use is used when writing to existing delta table and not while converting non-delta table to delta table.with error it is quite evident that your parquet files does not conform to unified scheme hence conversion fails.
lets try to skip automatic scheam inference by suplying the schema like this:
schema = "col1 INT, col2 STRING, col3 DOUBLE"
df = spark.read.schema(schema).parquet("
abfss://container@storage_account.dfs.core.windows.net/directory_name")and then write to delta:
df.write.format("delta").mode("overwrite").save("abfss://.../delta_table")
share your outcome. thanks
