Gepap
New Contributor II

The dataframe to write needs to have the following schema:

Column                             |  Type
----------------------------------------------
body (required)               |  string or binary 
partitionId (*optional)     |  string 
partitionKey (*optional)  |  string

This worked for me (pyspark version):

df.withColumn('body', F.to_json(
       F.struct(*df.columns),
       options={"ignoreNullFields": False}))\
   .select('body')\
   .write\
   .format("eventhubs")\
   .options(**ehconf)\
   .save()