https://community.databricks.com/t5/data-engineering/add-custom-metadata-to-avro-file-with-pyspark/m-p/9182#M4645 <P>Hello, </P><P>i need to add a custom metadata into a avro file. The avro file containt data. </P><P>we have tried to use "option" within the write function but it's not taken without generated any error.</P><P>df.write.format("avro").option("avro.codec", "snappy").option(<B>"header", "metadata_key:metadata_value")</B>.mode("overwrite").save("/tmp/avro_with_metadata")</P><P></P><P>I'm seeking a solution to add a custom metadata into a data avro file.</P><P>Thanks, </P><P>Zakaria</P><P></P> Thu, 16 Feb 2023 19:03:51 GMT zak 2023-02-16T19:03:51Z https://community.databricks.com/t5/data-engineering/add-custom-metadata-to-avro-file-with-pyspark/m-p/9182#M4645 <P>Hello, </P><P>i need to add a custom metadata into a avro file. The avro file containt data. </P><P>we have tried to use "option" within the write function but it's not taken without generated any error.</P><P>df.write.format("avro").option("avro.codec", "snappy").option(<B>"header", "metadata_key:metadata_value")</B>.mode("overwrite").save("/tmp/avro_with_metadata")</P><P></P><P>I'm seeking a solution to add a custom metadata into a data avro file.</P><P>Thanks, </P><P>Zakaria</P><P></P> Thu, 16 Feb 2023 19:03:51 GMT https://community.databricks.com/t5/data-engineering/add-custom-metadata-to-avro-file-with-pyspark/m-p/9182#M4645 zak 2023-02-16T19:03:51Z https://community.databricks.com/t5/data-engineering/add-custom-metadata-to-avro-file-with-pyspark/m-p/9186#M4649 <P>thank you a lot for your answer, it's very helpful.</P><P>I have a additionnal question please, is it possible to add a avro binnary metada inside the json avro_schema?</P> Fri, 24 Feb 2023 17:08:02 GMT https://community.databricks.com/t5/data-engineering/add-custom-metadata-to-avro-file-with-pyspark/m-p/9186#M4649 zak 2023-02-24T17:08:02Z