topic Is it possible to disable retryWrites using .option()? in Data Engineering

Is it possible to disable retryWrites using .option()?

tigger — Fri, 12 Nov 2021 14:50:36 GMT

Hello everyone,

I'm trying to write to DocumentDB using org.mongodb.spark:mongo-spark-connector_2.12:3.0.1. The DocDB is version 4 which doesn't support Retryable Writes so I disabled the feature setting option "retryWrites" to "false" (also tried with False). However it didn't work. Do you know why?

uri = "mongodb://username:password@host.docdb.amazonaws.com:27017" 
(df.write.format("mongo")
 .option("uri", uri)
 .option("retryWrites", "false")  
 .option("database", "mydb")
 .option("collection", "employee")
 .mode("append")
 .save())
---
Command failed with error 301: Retryable writes are not supported

I tried to set the option directly on the uri like below, then it works, but I'd like to use .option() to set all the connection options. Is it possible?

uri = "mongodb://username:password@host.docdb.amazonaws.com:27017/?retryWrites=false"

Thanks!

Re: Is it possible to disable retryWrites using .option()?

Sajesh — Fri, 12 Nov 2021 16:16:09 GMT

Hi @Hugh Vo ,

I can't find retryWrites available as an option in the MongoDB connector: https://docs.mongodb.com/spark-connector/current/configuration/#input-configuration.

Looks like, it has to be passed as part of the URI.

Re: Is it possible to disable retryWrites using .option()?

tigger — Fri, 12 Nov 2021 17:00:33 GMT

Thanks @Sajesh Manakkunnath

Re: Is it possible to disable retryWrites using .option()?

Anonymous — Fri, 12 Nov 2021 22:26:26 GMT

@Hugh Vo - If Sajehs's answer resolved the issue, would you be happy to mark their answer as best?