Hi Databricks Community,

I ran into the following issue when setting up a new cluster with the latest LTS Databricks runtime (11.3). When trying to install the package with the coordinates com.microsoft.azure.kusto:kusto-spark_3.0_2.12:3.1.4 from Maven, the install fails and I get the following error:

Library installation attempted on the driver node of cluster 1013-133741-jzb53a8t and failed. Please refer to the following error message to fix the library or contact Databricks support. Error Code: DRIVER_LIBRARY_INSTALLATION_FAILURE. Error Message: java.util.concurrent.ExecutionException: java.io.FileNotFoundException: File file:/local_disk0/tmp/clusterWideResolutionDir/maven/ivy/jars/io.netty_netty-transport-native-kqueue-4.1.59.Final.jar does not exist

This issue occurs for more recent versions of the package as well, e.g., com.microsoft.azure.kusto:kusto-spark_3.0_2.12:3.1.10.

What have I already tried?

• When using an older runtime, i.e., 10.4 LTS, this issue does not occur and the package installs succesfully.
• When installing the package using a jar file from the project's repository (https://github.com/Azure/azure-kusto-spark/releases) the install succeeds no matter which package or runtime was used, which makes me think that the issue is not with the package.

Any help in resolving this issue is greatly appreciated.

Kind regards,

Andrei