topic Re: How to integrate java 11 code in Databricks in Data Engineering

How to integrate java 11 code in Databricks

data_serf — Thu, 04 Aug 2022 21:17:40 GMT

Hi all,

We're trying to attach java libraries which are compiled/packaged using Java 11.

After doing some research it looks like even the most recent runtimes use Java 8 which can't run the Java 11 code ("wrong version 55.0, should be 52.0" errors)

Is there some way to work around this? Perhaps wrapping the java 11 lib code in Scala (which seems to be on a much more modern runtime?)

Also, in a longer term, is there anything on the roadmap for supporting a more modern version of Java?

Thank you!

Re: How to integrate java 11 code in Databricks

alexott — Fri, 05 Aug 2022 07:10:49 GMT

There is an experimental support for Java 11 starting with DBR 10.0 (release notes) - to activate it you need to put into environment variables configuration following value.

JNAME=zulu11-ca-amd64

Re: How to integrate java 11 code in Databricks

Vidula — Wed, 07 Sep 2022 11:53:23 GMT

Hi there @Andy Xue

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help.

We'd love to hear from you.

Thanks!

Re: How to integrate java 11 code in Databricks

matthewrj — Fri, 16 Sep 2022 03:28:43 GMT

I have tried setting JNAME=zulu11-ca-amd64 under Cluster > Advanced options > Spark > Environment variables but it doesn't seem to work. I still get errors indicating Java 8 is the JRE and in the Spark UI under "Environment" I still see:

Java Home: /usr/lib/jvm/zulu8-ca-amd64/jre

Java Version: 1.8.0_302 (Azul Systems, Inc.)

I am using cluster version "11.2.x-scala2.12".