add custom metadata to avro file with pyspark

[zak]

Hello,

i need to add a custom metadata into a avro file. The avro file containt data.

we have tried to use "option" within the write function but it's not taken without generated any error.

df.write.format("avro").option("avro.codec", "snappy").option("header", "metadata_key:metadata_value").mode("overwrite").save("/tmp/avro_with_metadata")

I'm seeking a solution to add a custom metadata into a data avro file.

Thanks,

Zakaria

[zak]

thank you a lot for your answer, it's very helpful.

I have a additionnal question please, is it possible to add a avro binnary metada inside the json avro_schema?