cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
JaydeepKhatri
by • New Contributor II
  • 1242 Views
  • 2 replies
  • 1 kudos

Resolved! Using merge Schema with spark.read.csv for inconsistent schemas

The Problem:A common data engineering challenge is reading a directory of CSV files where the schemas are inconsistent. For example, some files might have columns in a different order, or be missing certain columns altogether. The standard behavior o...

Get Started Discussions
Reply
  • 1242 Views
  • 2 replies
  • 1 kudos
View Replies
Latest Reply
Louis_Frolio
Databricks Employee
  • 1 kudos

  Hey @JaydeepKhatri  here are some helpful points to consider: Is this an officially supported, enhanced feature of the Databricks CSV reader? Based on internal research, this appears to be an undocumented “feature” of Spark running on Databricks. ...

  • 1 kudos
1 More Replies
Labels