Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-10-2021 01:37 AM
It looks like mergeschema is not active.
You can try to set the parameter active for the session with spark.databricks.delta.schema.automerge.enabled. That is for the whole sparksession.
A better solution is to pass a schema to your csv file (column names and types).
mergeSchema is pretty interesting , but it does not always work or give the results you want.