Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-21-2020 08:59 AM
I resolved it using the function monotonically_increasing_id and a little logic to set the Column Name.
To do this is necessary Java 1.8, because raise a error on function 'collect()' in Java 11.
df = df.withColumn('index', F.monotonically_increasing_id())
cols = df.columns
values = df.filter('index = 0').collect() # here define the skipped lines
for i in range(len(cols)):
if cols[i] != 'index':
df = df.select(df.columns).withColumnRenamed(cols[i], values[0][i])