How can I write Pandas dataframe into avro
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-07-2019 03:39 AM
I am trying to write Pandas core dataframe into avro format as below. But I get the following error:
AttributeError: 'DataFrame' object has no attribute 'write'
I have tried several options as below:
df_2018_pd.write.format("com.databricks.spark.avro").save("s3://folder/sub-folder/avro") df_2018_pd.write.format("avro").save("s3://folder/sub-folder/avro") df_2018_pd.write().format("com.databricks.spark.avro").save("s3://folder/sub-folder/avro") df_2018_pd.format("avro").save("s3://folder/sub-folder/avro")
All of the above gives error.
Any help would be appreciated.