I am able to connect to the cluster, browse its hive catalog, see tables/views and columns/datatypes

Running a simple select statement from a view on a parquet file produces this error and no other results:

"SQL Error [500540] [HY000]: [Databricks][DatabricksJDBCDriver](500540) Error caught in BackgroundFetcher. Foreground thread ID: 180. Background thread ID: 223. Error caught: sun.misc.Unsafe or java.nio.DirectByteBuffer.<init>(long, int) not available."

Standard Databricks cluster:

Standard_DS3_v2

Advanced Options Spark Config:

spark.databricks.cluster.profile singleNode

spark.databricks.io.directoryCommit.createSuccessFile false

spark.master local[*, 4]

spark.driver.extraJavaOptions -Dio.netty.tryReflectionSetAccessible=true

spark.hadoop.fs.azure.account.key.<reducted>.blob.core.windows.net <reducted>

spark.executor.extraJavaOptions -Dio.netty.tryReflectionSetAccessible=true

parquet.enable.summary-metadata false

My local machine:

Dbeaver Version 22.1.2.202207091909

Dbeaver JDBC URL:

jdbc:databricks://<reducted>.1.azuredatabricks.net:443/default;transportMode=http;ssl=1;httpPath=sql/protocolv1/o/<reducted>/<reducted>;AuthMech=3;UID=token;PWD=<reducted>

MacOS version (M1 chip): Monterey 12.4

Java version:

java --version

openjdk 18.0.1 2022-04-19

OpenJDK Runtime Environment Homebrew (build 18.0.1+0)

OpenJDK 64-Bit Server VM Homebrew (build 18.0.1+0, mixed mode, sharing)

I am able to do the following with no errors (Databricks default test dataset):

CREATE TABLE diamonds USING CSV OPTIONS (path "/databricks-datasets/Rdatasets/data-001/csv/ggplot2/diamonds.csv", header "true");

When I run this "select color from diamonds;" or this "select * from diamonds;"

I get this:

"SQL Error [500618] [HY000]: [Databricks][DatabricksJDBCDriver](500618) Error occured while deserializing arrow data: sun.misc.Unsafe or java.nio.DirectByteBuffer.<init>(long, int) not available"

Hence, any select query on any object (parquet file or anything else) causes the error described above.

What could be the problem? Any recommendations how to resolve this error? Why am I able to connect and see the metadata of the schemas/tables/views/columns, but not query or view the data?

P.S. I followed this guide exactly: https://docs.microsoft.com/en-us/azure/databricks/dev-tools/dbeaver#step-3-connect-dbeaver-to-your-a...