FabioKfouri
New Contributor II

I resolved it using the function monotonically_increasing_id and a little logic to set the Column Name.

To do this is necessary Java 1.8, because raise a error on function 'collect()' in Java 11.

df = df.withColumn('index', F.monotonically_increasing_id())
cols = df.columns
values = df.filter('index = 0').collect()  # here define the skipped lines
for i in range(len(cols)):
    if cols[i] != 'index':
        df = df.select(df.columns).withColumnRenamed(cols[i], values[0][i])