Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-31-2025 08:48 AM
I am not sure if I read the full explanation but how about this :
df
.withColumn('value_str', F.decode(F.col('value'), 'utf-8'))
.withColumn('value_json', F.explode(F.from_json(F.col('value_str'), json_schema)))
.select('value_json.*')
.select(F.col('user_id').alias('userid'),
F.col('platform').alias('platform'),
F.col('event_id').alias('eventid'),
F.col('event_date').alias('eventdate'))
