โ09-03-2022 09:18 AM
Connecting to Databricks using OpenJDK 17 I got the exception below. Are there any plans to fix the driver for OpenJDK17?
java.sql.SQLException: [Databricks][DatabricksJDBCDriver](500540) Error caught in BackgroundFetcher. Foreground thread ID: 44. Background thread ID: 46. Error caught: null.
at com.databricks.client.hivecommon.dataengine.BackgroundFetcher.run(Unknown Source)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
com.databricks.client.support.exceptions.GeneralException: [Databricks][DatabricksJDBCDriver](500540) Error caught in BackgroundFetcher. Foreground thread ID: 44. Background thread ID: 46. Error caught: null.
at com.databricks.client.hivecommon.dataengine.BackgroundFetcher.run(Unknown Source)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
java.lang.ExceptionInInitializerError
at com.databricks.client.jdbc42.internal.apache.arrow.memory.ArrowBuf.getDirectBuffer(ArrowBuf.java:228)
at com.databricks.client.jdbc42.internal.apache.arrow.memory.ArrowBuf.nioBuffer(ArrowBuf.java:223)
at com.databricks.client.jdbc42.internal.apache.arrow.vector.ipc.ReadChannel.readFully(ReadChannel.java:87)
at com.databricks.client.jdbc42.internal.apache.arrow.vector.ipc.message.MessageSerializer.readMessageBody(MessageSerializer.java:727)
at com.databricks.client.jdbc42.internal.apache.arrow.vector.ipc.message.MessageSerializer.deserializeRecordBatch(MessageSerializer.java:363)
at com.databricks.client.spark.arrow.ArrowBuffer.deserializeBatch(Unknown Source)
at com.databricks.client.spark.arrow.ArrowBuffer.handleInitializeBuffer(Unknown Source)
at com.databricks.client.hivecommon.api.HiveServer2BaseBuffer.initializeBuffer(Unknown Source)
at com.databricks.client.hivecommon.api.RowsetBuffer.initializeBuffer(Unknown Source)
at com.databricks.client.hivecommon.api.HS2Client.getRowSetInformation(Unknown Source)
at com.databricks.client.hivecommon.api.HS2Client.fetchFromServer(Unknown Source)
at com.databricks.client.spark.jdbc.DowloadableFetchClient.fetchNRows(Unknown Source)
at com.databricks.client.hivecommon.api.HS2Client.fetchRows(Unknown Source)
at com.databricks.client.hivecommon.dataengine.BackgroundFetcher.run(Unknown Source)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
โ12-12-2022 07:44 AM
@Dragos Pruteanuโ Not sure if you are still having trouble with this, but the issues for me seemed to be related to arrow serialization which can be disabled by adding "EnableArrow=0" to your JDBC URL.
โ09-07-2022 01:58 AM
Hi @Dragos Pruteanuโ, This article describes how to configure the Databricks ODBC and JDBC drivers to connect your tools or clients to Azure Databricks.
โ09-07-2022 10:25 PM
Thank you for sending the link. But the driver fails to work if the client is using JDK 17 or higher. This is a compatibility issue with the latest Java releases.
โ12-12-2022 07:44 AM
@Dragos Pruteanuโ Not sure if you are still having trouble with this, but the issues for me seemed to be related to arrow serialization which can be disabled by adding "EnableArrow=0" to your JDBC URL.
โ03-10-2023 11:12 AM
Thanks,
โ08-01-2023 02:23 AM
I still see the above error with databricks jdbc driver 2.6.33. Anyone aware of fix available either in driver or java?
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
Click here to register and join today!
Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.