topic Getting connection reset issue while connecting to a SQL server in Data Engineering

Getting connection reset issue while connecting to a SQL server

van45678 — Wed, 19 Mar 2025 10:40:54 GMT

Hello All,

I am unable to connect to a SQL server instance that is installed in a on-premise network from databricks. I am able to successfully ping the server from the notebook using this command [nc -vz <hostname> <port>] which means I am able to establish a connection.

However when I run the following command, it only runs into a SocketException. Any input on this would be greatly appreciated.

driver = "com.microsoft.sqlserver.jdbc.SQLServerDriver"
jdbcHostname = '<host>'
jdbcPort = <port>
jdbcDatabase = '<database>'
user = "<username>"
password = "<password>"
url = "jdbc:sqlserver://{0}:{1};database={2};encrypt=false;user={3};password={4}".format(jdbcHostname, jdbcPort, jdbcDatabase, user, password)

test_query = "select * from <tablename>"
df = spark.read \
.format("jdbc") \
.option("driver", driver) \
.option("url", url) \
.option("query", test_query) \
.load()
df.printschema()
df.show(1)

Here are the details of exception -

Py4JJavaError: An error occurred while calling o407.load.
: com.microsoft.sqlserver.jdbc.SQLServerException: Connection reset ClientConnectionId:d29245e8-6e7b-4ed3-bb32-a514aba5a60b
at com.microsoft.sqlserver.jdbc.SQLServerConnection.terminate(SQLServerConnection.java:3806)
at com.microsoft.sqlserver.jdbc.TDSChannel.read(IOBuffer.java:2109)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.prelogin(SQLServerConnection.java:3517)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3325)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:2950)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:2790)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1663)
at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1064)
at org.apache.spark.sql.execution.datasources.jdbc.connection.BasicConnectionProvider.getConnection(BasicConnectionProvider.scala:49)
at org.apache.spark.sql.execution.datasources.jdbc.connection.ConnectionProviderBase.create(ConnectionProvider.scala:102)
at org.apache.spark.sql.jdbc.JdbcDialect.$anonfun$createConnectionFactory$1(JdbcDialects.scala:168)
at org.apache.spark.sql.jdbc.JdbcDialect.$anonfun$createConnectionFactory$1$adapted(JdbcDialects.scala:164)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.getQueryOutputSchema(JDBCRDD.scala:83)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:78)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCRelation$.getSchema(JDBCRelation.scala:241)
at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:43)
at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:391)
at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:381)
at org.apache.spark.sql.DataFrameReader.$anonfun$load$2(DataFrameReader.scala:337)
at scala.Option.getOrElse(Option.scala:189)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:337)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:227)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:397)
at py4j.Gateway.invoke(Gateway.java:306)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:199)
at py4j.ClientServerConnection.run(ClientServerConnection.java:119)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:210)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at com.microsoft.sqlserver.jdbc.TDSChannel$ProxyInputStream.readInternal(IOBuffer.java:1207)
at com.microsoft.sqlserver.jdbc.TDSChannel$ProxyInputStream.read(IOBuffer.java:1194)
at com.microsoft.sqlserver.jdbc.TDSChannel.read(IOBuffer.java:2100)

Re: Getting connection reset issue while connecting to a SQL server

Kebadu — Tue, 29 Apr 2025 17:38:06 GMT

Hi Run in to similar problem. Were you able to find a solution?

Thanks

Re: Getting connection reset issue while connecting to a SQL server

mark_ott — Sun, 09 Nov 2025 14:28:54 GMT

The error you are encountering, "com.microsoft.sqlserver.jdbc.SQLServerException: Connection reset," even after a successful nc (netcat) connection, is a common but nuanced problem when connecting Databricks to an on-premise SQL Server. Although your notebook can ping the server and connect at the TCP/IP level, the JDBC handshake can still be interrupted or reset due to issues at the protocol, authentication, or firewall/proxy layer.

Key Troubleshooting Steps

Authentication Issues
- Ensure that the username and password are valid for SQL Server authentication (not Windows Auth or Active Directory unless explicitly supported and configured).
- Confirm that SQL Authentication is enabled on the SQL Server.
JDBC URL and Encryption
- If encrypt=false was specified but SQL Server enforces SSL or vice versa, this can trigger a reset.
- Try connecting with encrypt=true;trustServerCertificate=true in the JDBC string (unless your company policy or server prohibits this).
- Test with and without the encryption parameters, based on what the server expects.
Firewall and Network Path
- Even if ping/nc is successful, some firewalls allow TCP handshakes but block or reset application-layer traffic, especially over certain ports or for long-lived connections.
- Verify with your network team that the SQL Server port (default 1433) allows inbound JDBC traffic from your Databricks IPs for the full session duration.
- Check for any "stateful inspection" or IDS/IPS devices that could drop or reset connections.
SQL Server Configuration
- Make sure TCP/IP is enabled on the SQL Server (SQL Server Configuration Manager > SQL Server Network Configuration > Protocols for [Instance]).
- Confirm that your SQL Server is bound to the correct IP or 'All IPs', and not 'localhost' only.
JDBC Driver Version
- Ensure you are using a compatible and up-to-date Microsoft JDBC driver for SQL Server. Old or mismatched drivers can cause handshake failures.
Proxy/Gateway Issues
- If connectivity to on-premise is via VPN, SSH tunnel, or another proxy, check that session timeouts/connection limits are not interfering.

Additional Suggestions

Use a simple test query (like SELECT 1) to rule out data or schema-related issues.
Review SQL Server error logs for any incoming connection failures or network-level rejections.
If possible, try the same JDBC connection from a different system on the same network to isolate if the problem is Databricks-specific.
If your server enforces SSL and you provide encrypt=false, or vice versa, a reset is likely.
Some enterprise networks block dynamic port allocation; ensure the SQL Server is using a static port.

Example JDBC URLs

For non-SSL (if server allows):

text

jdbc:sqlserver://hostname:1433;database=dbname;encrypt=false;user=user;password=pass

For SSL (try this if non-SSL does not work):

text

jdbc:sqlserver://hostname:1433;database=dbname;encrypt=true;trustServerCertificate=true;user=user;password=pass

If custom instance (not the default), specify instanceName=....

What to Check Next

Confirm the correct SQL authentication and connection parameters.
Test with both encrypt=false and encrypt=true;trustServerCertificate=true.
Review SQL Server and network/firewall logs for clues.
Ask your network/security team if a proxy, NAT, or VPN is terminating the session unexpectedly.

Most likely, the issue is not basic TCP reachability but protocol-level, authentication, or enforced encryption mismatch. Work with your SQL Server and network teams to ensure all layers are compatible with your Databricks configurationration.