topic mongodb ingest data into databricks error in Data Engineering https://community.databricks.com/t5/data-engineering/mongodb-ingest-data-into-databricks-error/m-p/97466#M39486 <DIV><DIV><SPAN>spark </SPAN><SPAN>=</SPAN><SPAN> SparkSession.builder \</SPAN></DIV><DIV><SPAN>.</SPAN><SPAN>appName</SPAN><SPAN>(</SPAN><SPAN>"MongoDBToDatabricks"</SPAN><SPAN>) \</SPAN></DIV><DIV><SPAN>.</SPAN><SPAN>config</SPAN><SPAN>(</SPAN><SPAN>"spark.jars.packages"</SPAN><SPAN>, </SPAN><SPAN>"org.mongodb.spark:mongo-spark-connector_2.12:10.4.0"</SPAN><SPAN>) \</SPAN></DIV><DIV><SPAN>.</SPAN><SPAN>config</SPAN><SPAN>(</SPAN><SPAN>"spark.mongodb.read.connection.uri"</SPAN><SPAN>, mongodb_uri) \</SPAN></DIV><DIV><SPAN>.</SPAN><SPAN>config</SPAN><SPAN>(</SPAN><SPAN>"spark.mongodb.write.connection.uri"</SPAN><SPAN>, mongodb_uri) \</SPAN></DIV><DIV><SPAN>.</SPAN><SPAN>getOrCreate</SPAN><SPAN>()</SPAN></DIV><BR /><DIV><SPAN># Read data from MongoDB</SPAN></DIV><DIV><SPAN>df </SPAN><SPAN>=</SPAN><SPAN> spark.read \</SPAN></DIV><DIV><SPAN>.</SPAN><SPAN>format</SPAN><SPAN>(</SPAN><SPAN>"com.mongodb.spark.sql.connector.MongoTableProvider"</SPAN><SPAN>) \</SPAN></DIV><DIV><SPAN>.</SPAN><SPAN>option</SPAN><SPAN>(</SPAN><SPAN>"database"</SPAN><SPAN>, database_name) \</SPAN></DIV><DIV><SPAN>.</SPAN><SPAN>option</SPAN><SPAN>(</SPAN><SPAN>"collection"</SPAN><SPAN>, collection_name) \</SPAN></DIV><DIV><SPAN>.</SPAN><SPAN>option</SPAN><SPAN>(</SPAN><SPAN>"readPreference"</SPAN><SPAN>, </SPAN><SPAN>"primary"</SPAN><SPAN>) \</SPAN></DIV><DIV><SPAN>.</SPAN><SPAN>load</SPAN><SPAN>()</SPAN></DIV><BR /><DIV><SPAN>df.</SPAN><SPAN>printSchema</SPAN><SPAN>()</SPAN></DIV></DIV> Mon, 04 Nov 2024 08:37:20 GMT jeft 2024-11-04T08:37:20Z mongodb ingest data into databricks error https://community.databricks.com/t5/data-engineering/mongodb-ingest-data-into-databricks-error/m-p/97466#M39486 <DIV><DIV><SPAN>spark </SPAN><SPAN>=</SPAN><SPAN> SparkSession.builder \</SPAN></DIV><DIV><SPAN>.</SPAN><SPAN>appName</SPAN><SPAN>(</SPAN><SPAN>"MongoDBToDatabricks"</SPAN><SPAN>) \</SPAN></DIV><DIV><SPAN>.</SPAN><SPAN>config</SPAN><SPAN>(</SPAN><SPAN>"spark.jars.packages"</SPAN><SPAN>, </SPAN><SPAN>"org.mongodb.spark:mongo-spark-connector_2.12:10.4.0"</SPAN><SPAN>) \</SPAN></DIV><DIV><SPAN>.</SPAN><SPAN>config</SPAN><SPAN>(</SPAN><SPAN>"spark.mongodb.read.connection.uri"</SPAN><SPAN>, mongodb_uri) \</SPAN></DIV><DIV><SPAN>.</SPAN><SPAN>config</SPAN><SPAN>(</SPAN><SPAN>"spark.mongodb.write.connection.uri"</SPAN><SPAN>, mongodb_uri) \</SPAN></DIV><DIV><SPAN>.</SPAN><SPAN>getOrCreate</SPAN><SPAN>()</SPAN></DIV><BR /><DIV><SPAN># Read data from MongoDB</SPAN></DIV><DIV><SPAN>df </SPAN><SPAN>=</SPAN><SPAN> spark.read \</SPAN></DIV><DIV><SPAN>.</SPAN><SPAN>format</SPAN><SPAN>(</SPAN><SPAN>"com.mongodb.spark.sql.connector.MongoTableProvider"</SPAN><SPAN>) \</SPAN></DIV><DIV><SPAN>.</SPAN><SPAN>option</SPAN><SPAN>(</SPAN><SPAN>"database"</SPAN><SPAN>, database_name) \</SPAN></DIV><DIV><SPAN>.</SPAN><SPAN>option</SPAN><SPAN>(</SPAN><SPAN>"collection"</SPAN><SPAN>, collection_name) \</SPAN></DIV><DIV><SPAN>.</SPAN><SPAN>option</SPAN><SPAN>(</SPAN><SPAN>"readPreference"</SPAN><SPAN>, </SPAN><SPAN>"primary"</SPAN><SPAN>) \</SPAN></DIV><DIV><SPAN>.</SPAN><SPAN>load</SPAN><SPAN>()</SPAN></DIV><BR /><DIV><SPAN>df.</SPAN><SPAN>printSchema</SPAN><SPAN>()</SPAN></DIV></DIV> Mon, 04 Nov 2024 08:37:20 GMT https://community.databricks.com/t5/data-engineering/mongodb-ingest-data-into-databricks-error/m-p/97466#M39486 jeft 2024-11-04T08:37:20Z Re: mongodb ingest data into databricks error https://community.databricks.com/t5/data-engineering/mongodb-ingest-data-into-databricks-error/m-p/97469#M39488 <P><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="Screenshot 2024-11-04 at 4.34.15 PM.png" style="width: 400px;"><img src="https://community.databricks.com/t5/image/serverpage/image-id/12566i81202104C500487B/image-size/medium?v=v2&amp;px=400" role="button" title="Screenshot 2024-11-04 at 4.34.15 PM.png" alt="Screenshot 2024-11-04 at 4.34.15 PM.png" /></span></P><P> it shows maximun number of retires has been exceeded. i have no idea why it couldnt connect as i able to telnet to mongoDB</P> Mon, 04 Nov 2024 08:40:33 GMT https://community.databricks.com/t5/data-engineering/mongodb-ingest-data-into-databricks-error/m-p/97469#M39488 jeft 2024-11-04T08:40:33Z Re: mongodb ingest data into databricks error https://community.databricks.com/t5/data-engineering/mongodb-ingest-data-into-databricks-error/m-p/100212#M40226 <P>Hello&nbsp;<a href="https://community.databricks.com/t5/user/viewprofilepage/user-id/130990">@jeft</a>&nbsp;, will you be able to share some screenshots of the driver logs?</P> Wed, 27 Nov 2024 10:40:12 GMT https://community.databricks.com/t5/data-engineering/mongodb-ingest-data-into-databricks-error/m-p/100212#M40226 Nam_Nguyen 2024-11-27T10:40:12Z