Write streaming output to DynamoDB
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-20-2022 08:52 AM
Hi All,
I am trying to write a streaming DF into dynamoDB with below code.
tumbling_df.writeStream \
.format("org.apache.spark.sql.execution.streaming.sinks.DynamoDBSinkProvider") \
.option("region", "eu-west-2") \
.option("tableName", "PythonForeachTest") \
.outputMode("update") \
.start()
However i get an error
Py4JJavaError: An error occurred while calling o3880.start.
: java.lang.ClassNotFoundException:
Failed to find data source: org.apache.spark.sql.execution.streaming.sinks.DynamoDBSinkProvider. Please find packages at
http://spark.apache.org/third-party-projects.html
Could you please help me to find this package and how to import it please?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-20-2022 12:15 PM
Hi @SUDHANSHU RAJ ,
I can't seem to find much on the "DynamoDBSinkProvider" source. Have you checked out the link for the streaming to DynamoDB documentation?