Yes, did not find official way to report bugs directly to Databricks, but it would be nice if some databricks engineer could open a respective ticket in an internal Jira.
In our case this is pretty much a show stopper for reading data exported from client SAP systems, as the exported data contains 8 header rows some of which are empty (only contain tabs).
What we planned to do was:
- read the first header row
- skip the remaining 7 header rows
But if one of the headers is empty, it is already skipped by spark and we skip the first row of real data.