cancel
Showing results for 
Search instead for 
Did you mean: 
Community Platform Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Showing results for 
Search instead for 
Did you mean: 

Intermittent (cert) failure when connecting to AWS RDS

CraiMacl_23588
New Contributor

I've just upgraded a bunch of jobs to 12.2 LTS runtime and now getting intermittent failures with the following message:

```

java.sql.SQLException: [Amazon](600000) Error setting/closing connection: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target.

```

We are using pyspark to connect (cross-account) to and RDS instance. This was working absolutely fine on runtime 7.3, and works some of the time on 12.2. However, some of our scheduled jobs are failing.

Waiting a while, restarting the cluster and retrying can make the job succeed.

I can see another couple of people have had the same issue:
* https://community.databricks.com/t5/warehousing-analytics/trying-to-connect-to-dbeaver-from-databric...

https://community.databricks.com/t5/data-engineering/the-driver-could-not-establish-a-secure-connect...

Any ideas what's going wrong?

1 REPLY 1

Kaniz_Fatma
Community Manager
Community Manager

Hi @CraiMacl_23588This error is related to an SSL certificate validation failure.

To resolve the issue, you can try the following steps:

• Check if the RDS instance SSL certificate is valid and not expired.

• Ensure the root and intermediate certificates are installed on the client machine.

• If the certificates are not installed, download them from the RDS console and install them on the client machine.

• If the certificates are installed, check if the certificate chain is complete and valid.

• If the certificate chain is not complete or valid, install the missing certificates.

• If the issue persists, try increasing the spark.network.timeout configuration value to a higher value than the default of 120 seconds. The value can be set at the cluster level or the notebook level.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group