How to ensure all resources are released when closing a Connection via DataBricks JDBC Driver?

[jon1]

Hi there,

I'm building a Sink that uses the Databricks JDBC Driver version 2.6.27. It appears that regardless of using a try-with-resources block and explicitly closing the Connection, the driver's internals are not releasing IdleConnectionEvictor threads which are spawned per Connection.

Connection.close() doesn't release the IdleConnectionEvictor thread:

app//com.databricks.client.jdbc42.internal.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)

So if you're spawning multiple Connections, you're left with these threads until Java process stop and restart.

Is this a bug or am I overlooking a step required by the DataBricks JDBC Driver?

P.S: I'd post code but no markdown syntax seems to work in this DataBricks Community forum for me. There's no code formatting options when starting a new discussion either.

[jon1]

Following appears to be have been resolved in JDBC driver version 2.6.29 and newer. See release notes and the bug fix [SPARKJ-614] After closing HTTP connections, the driver does not clean up native threads.

[RakeshSen01]

Still not fixed got 14k instances of com.databricks.client.jdbc42.internal.apache.http.impl.client.IdleConnectionEvictor 14,037 (0%) 673,776 B (0%) n/a And due to that application got to crashed. can you please help out as this is blocking to release databricks feature.