09-03-2022 09:18 AM
Connecting to Databricks using OpenJDK 17 I got the exception below. Are there any plans to fix the driver for OpenJDK17?
java.sql.SQLException: [Databricks][DatabricksJDBCDriver](500540) Error caught in BackgroundFetcher. Foreground thread ID: 44. Background thread ID: 46. Error caught: null.
at com.databricks.client.hivecommon.dataengine.BackgroundFetcher.run(Unknown Source)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
com.databricks.client.support.exceptions.GeneralException: [Databricks][DatabricksJDBCDriver](500540) Error caught in BackgroundFetcher. Foreground thread ID: 44. Background thread ID: 46. Error caught: null.
at com.databricks.client.hivecommon.dataengine.BackgroundFetcher.run(Unknown Source)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
java.lang.ExceptionInInitializerError
at com.databricks.client.jdbc42.internal.apache.arrow.memory.ArrowBuf.getDirectBuffer(ArrowBuf.java:228)
at com.databricks.client.jdbc42.internal.apache.arrow.memory.ArrowBuf.nioBuffer(ArrowBuf.java:223)
at com.databricks.client.jdbc42.internal.apache.arrow.vector.ipc.ReadChannel.readFully(ReadChannel.java:87)
at com.databricks.client.jdbc42.internal.apache.arrow.vector.ipc.message.MessageSerializer.readMessageBody(MessageSerializer.java:727)
at com.databricks.client.jdbc42.internal.apache.arrow.vector.ipc.message.MessageSerializer.deserializeRecordBatch(MessageSerializer.java:363)
at com.databricks.client.spark.arrow.ArrowBuffer.deserializeBatch(Unknown Source)
at com.databricks.client.spark.arrow.ArrowBuffer.handleInitializeBuffer(Unknown Source)
at com.databricks.client.hivecommon.api.HiveServer2BaseBuffer.initializeBuffer(Unknown Source)
at com.databricks.client.hivecommon.api.RowsetBuffer.initializeBuffer(Unknown Source)
at com.databricks.client.hivecommon.api.HS2Client.getRowSetInformation(Unknown Source)
at com.databricks.client.hivecommon.api.HS2Client.fetchFromServer(Unknown Source)
at com.databricks.client.spark.jdbc.DowloadableFetchClient.fetchNRows(Unknown Source)
at com.databricks.client.hivecommon.api.HS2Client.fetchRows(Unknown Source)
at com.databricks.client.hivecommon.dataengine.BackgroundFetcher.run(Unknown Source)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
12-12-2022 07:44 AM
@Dragos Pruteanu Not sure if you are still having trouble with this, but the issues for me seemed to be related to arrow serialization which can be disabled by adding "EnableArrow=0" to your JDBC URL.
09-07-2022 10:25 PM
Thank you for sending the link. But the driver fails to work if the client is using JDK 17 or higher. This is a compatibility issue with the latest Java releases.
12-12-2022 07:44 AM
@Dragos Pruteanu Not sure if you are still having trouble with this, but the issues for me seemed to be related to arrow serialization which can be disabled by adding "EnableArrow=0" to your JDBC URL.
03-10-2023 11:12 AM
Thanks,
08-01-2023 02:23 AM
I still see the above error with databricks jdbc driver 2.6.33. Anyone aware of fix available either in driver or java?
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group