User16857281974
Databricks Employee
Databricks Employee

First you are bouncing between the RDD and DataFrames API. If you start with a SparkSession object from the DF API instead you can make the call

spark.read.option("header", "true").csv(path)

then you don't have to split the file, convert from RDD to DF and the first column will be read as a header instead of as data.

For more information you can also look at the ~20 other options available to the DataFrameReader (spark.read)