Hi. I am trying to read from our Microsoft SQL Server from Azure Databricks via spark.read.jdbc() as described here: Query databases using JDBC - Azure Databricks | Microsoft Learn. The SQL Server is on an Azure VM in a virtual network peered with the virtual network of the azure databricks workspace.
The reading/writing used to work without problems. Last week, I started to get the error message when manually triggering the function in a notebook:
com.microsoft.sqlserver.jdbc.SQLServerException: The driver could not establish a secure connection to SQL Server by using Secure Sockets Layer (SSL) encryption. Error: "PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target".
The weird thing is: In my automated databricks workflow I also use read/write operations. The first read and write operation work as usual, but the second read operation is stuck without throwing an error. If I don't manually cancel the run, the function will just be stuck for hours.
Would be glad if sombody could give me a hint at what the problem could be?