Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-17-2022 08:47 AM
The dataframe to write needs to have the following schema:
Column | Type
----------------------------------------------
body (required) | string or binary
partitionId (*optional) | string
partitionKey (*optional) | stringThis worked for me (pyspark version):
df.withColumn('body', F.to_json(
F.struct(*df.columns),
options={"ignoreNullFields": False}))\
.select('body')\
.write\
.format("eventhubs")\
.options(**ehconf)\
.save()