I came across the same issue and the file causing problems needed the csv option "multiline" set back to the default "false" to read the file:

df = spark.read.option("multiline", "false").csv("CSV_PATH")

 If this approach eliminates the error above, I would still recommend validating that your data is read in correctly with this option change before considering it resolved.