- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-29-2023 03:29 PM
Few points
1. chce if you installed exactly same driver version as you are pointing this in code (2.12:3.2.0) it has to match 100percent
org.mongodb.spark:mongo-spark-connector_2.12:3.2.0
2. I have seen people configuring connction to atlas in two ways
Option 1
Back in Databricks in your cluster configuration, under Advanced Options (bottom of page), paste the connection string for both the spark.mongodb.output.uri and spark.mongodb.input.uri variables. Plase populate the username and password field appropriatly. This way all the workbooks you are running on the cluster will use this configuration.
Option 2:
Alternativley you can explictly set the option when calling APIs like: spark.read.format("mongo").option("spark.mongodb.input.uri", connectionString).load(). If congigured the variables in the cluster, you don't have to set the option.
3. Try with single user cluster ( if you are using shared cluster) .
4. Check mongo db driver compatibility with spark version you are using
Hope that it will help .