How can I write Pandas dataframe into avro

[sobhan]

I am trying to write Pandas core dataframe into avro format as below. But I get the following error:

AttributeError: 'DataFrame' object has no attribute 'write'

I have tried several options as below:

df_2018_pd.write.format("com.databricks.spark.avro").save("s3://folder/sub-folder/avro") df_2018_pd.write.format("avro").save("s3://folder/sub-folder/avro") df_2018_pd.write().format("com.databricks.spark.avro").save("s3://folder/sub-folder/avro") df_2018_pd.format("avro").save("s3://folder/sub-folder/avro")

All of the above gives error.

Any help would be appreciated.

[JosieMundi]

Not sure if you could use something like:

https://github.com/ynqa/pandavro

they have a to_avro which writes pandas to avro

[sobhan]

Yes i saw that pandavro pkg and installed with pip3 but still it did not resolve. So I had to convert into JSON dictionary object and proceed from there. I solved the problem not using Pandas dataframe into AVRO. May be some useful Pandas utility for this will come in future

Thanks for your response.

[Brayden_Cook]

Very complicated question. I think you can get your answer on online sites. There are many online providers like managements writing solutions whose experts provide online help for every type of research paper. I got a lot of assistance from them. Nowadays it has become very easy to solve any type of questions with the help online websites.