Intermittent (cert) failure when connecting to AWS RDS

[CraiMacl_23588]

I've just upgraded a bunch of jobs to 12.2 LTS runtime and now getting intermittent failures with the following message:

```

java.sql.SQLException: [Amazon](600000) Error setting/closing connection: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target.

```

We are using pyspark to connect (cross-account) to and RDS instance. This was working absolutely fine on runtime 7.3, and works some of the time on 12.2. However, some of our scheduled jobs are failing.

Waiting a while, restarting the cluster and retrying can make the job succeed.

I can see another couple of people have had the same issue:
* https://community.databricks.com/t5/warehousing-analytics/trying-to-connect-to-dbeaver-from-databric...

* https://community.databricks.com/t5/data-engineering/the-driver-could-not-establish-a-secure-connect...

Any ideas what's going wrong?

[Kaniz_Fatma]

Hi @CraiMacl_23588, This error is related to an SSL certificate validation failure.

To resolve the issue, you can try the following steps:

• Check if the RDS instance SSL certificate is valid and not expired.

• Ensure the root and intermediate certificates are installed on the client machine.

• If the certificates are not installed, download them from the RDS console and install them on the client machine.

• If the certificates are installed, check if the certificate chain is complete and valid.

• If the certificate chain is not complete or valid, install the missing certificates.

• If the issue persists, try increasing the spark.network.timeout configuration value to a higher value than the default of 120 seconds. The value can be set at the cluster level or the notebook level.