How to ensure all resources are released when closing a Connection via DataBricks JDBC Driver?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-09-2022 02:34 AM
Hi there,
I'm building a Sink that uses the Databricks JDBC Driver version 2.6.27. It appears that regardless of using a try-with-resources block and explicitly closing the Connection, the driver's internals are not releasing IdleConnectionEvictor threads which are spawned per Connection.
Connection.close() doesn't release the IdleConnectionEvictor thread:
app//com.databricks.client.jdbc42.internal.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
So if you're spawning multiple Connections, you're left with these threads until Java process stop and restart.
Is this a bug or am I overlooking a step required by the DataBricks JDBC Driver?
P.S: I'd post code but no markdown syntax seems to work in this DataBricks Community forum for me. There's no code formatting options when starting a new discussion either.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-19-2022 06:12 AM
Following appears to be have been resolved in JDBC driver version 2.6.29 and newer. See release notes and the bug fix [SPARKJ-614] After closing HTTP connections, the driver does not clean up native threads.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-13-2024 05:27 AM
Still not fixed got 14k instances of com.databricks.client.jdbc42.internal.apache.http.impl.client.IdleConnectionEvictor 14,037 (0%) 673,776 B (0%) n/a And due to that application got to crashed. can you please help out as this is blocking to release databricks feature.
