cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Issues when adding keystore spark config for pyspark to mongo atlas X.509 connectivity

jash281098
New Contributor II

Step followed - 

Step1: To add init script that will copy the keystore file in the tmp location.

Step2: To add spark config in cluster advance options - 

spark.driver.extraJavaOptions -Djavax.net.ssl.keyStore=/tmp/keystore.jks -Djavax.net.ssl.keyStorePassword=<pass>
spark.executor.extraJavaOptions -Djavax.net.ssl.keyStore=/tmp/keystore.jks -Djavax.net.ssl.keyStorePassword=<pass>

After doing above configuration the cluster doesn't execute any commands. In log4j driver logs their is a constant error message. What is the recommended way to achieve mongo atlas X.509 connectivity?

Error Could not connect to mdb7sywh50xhpr.chkweekm4xjq.us-east-1.rds.amazonaws.com:3306 : readHandshakeRecord

1 REPLY 1

jash281098
New Contributor II

Error Messages -

Caused by: com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Failed to initialize pool: Could not connect to address=(host=mdb7sywh50xhpr.chkweekm4xjq.us-east-1.rds.amazonaws.com)(port=3306)(type=master) : Could not connect to mdb7sywh50xhpr.chkweekm4xjq.us-east-1.rds.amazonaws.com:3306 : readHandshakeRecord

Caused by: java.net.SocketException: Broken pipe (Write failed)

25/05/10 14:19:06 WARN HiveClientImpl: HiveClient got thrift or connection reset exception, destroying client and retrying (13 tries remaining)

Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient

Caused by: javax.jdo.JDOFatalInternalException: Error creating transactional connection factory

Caused by: java.lang.reflect.InvocationTargetException

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now