topic Re: JDBC Error: Error occured while deserializing arrow data in Data Engineering

JDBC Error: Error occured while deserializing arrow data

mattmunz — Sat, 30 Jul 2022 02:15:25 GMT

I am getting the following error in my Java application.

java.sql.SQLException: [Databricks][DatabricksJDBCDriver](500618) Error occured while deserializing arrow data: sun.misc.Unsafe or java.nio.DirectByteBuffer.<init>(long, int) not available

I believe this is the same issue described here.

Java version:

java version "18.0.1.1" 2022-04-22
Java(TM) SE Runtime Environment (build 18.0.1.1+2-6)
Java HotSpot(TM) 64-Bit Server VM (build 18.0.1.1+2-6, mixed mode, sharing)

I am using the recommended flag:

-Dio.netty.tryReflectionSetAccessible=true

Is the source code for the JDBC driver available? Any suggestions on how to fix this would be appreciated. Also if there is an example app that uses the JDBC driver, I'd be interested in trying that as well. Thanks!

Re: JDBC Error: Error occured while deserializing arrow data

User16753725469 — Sun, 27 Nov 2022 19:27:25 GMT

Please try adding the below(--add-opens flag) java command line flags in your jvm call:

% javac -classpath SparkJDBC42Example.jar:. jdbc_example.java

% java --add-opens=java.base/java.nio=ALL-UNNAMED -classpath SparkJDBC42Example.jar:. jdbc_example

Re: JDBC Error: Error occured while deserializing arrow data

cvcore — Tue, 18 Feb 2025 08:27:45 GMT

For anyone encountering this issue in 2025, I was able to solve it by using the

--add-opens=jdk.unsupported/sun.misc=ALL-UNNAMED

option in combination with the latest jdbc driver (v2.7.1). I was using the driver in dbeaver, but I assume the issue could be also solved in similar environments.

Pointer: https://github.com/dbeaver/dbeaver/issues/18962