Hi,
I am trying to implement the Spark TableProvider api and i am experiencing a jar conflict (I am using the 17.3 runtime).
com.fasterxml.jackson.databind.JsonMappingException: Scala module 2.15.2 requires Jackson Databind version >= 2.15.0 and < 2.16.0 - Found jackson-databind version 2.18.2
at shaded.databricks.org.apache.hudi.com.fasterxml.jackson.module.scala.JacksonModule.setupModule(JacksonModule.scala:61)
at shaded.databricks.org.apache.hudi.com.fasterxml.jackson.module.scala.JacksonModule.setupModule$(JacksonModule.scala:46)
at shaded.databricks.org.apache.hudi.com.fasterxml.jackson.module.scala.DefaultScalaModule.setupModule(DefaultScalaModule.scala:17)
at com.fasterxml.jackson.databind.ObjectMapper.registerModule(ObjectMapper.java:908)
at com.fasterxml.jackson.databind.ObjectMapper.registerModules(ObjectMapper.java:1110)
at com.fasterxml.jackson.databind.ObjectMapper.findAndRegisterModules(ObjectMapper.java:1194)
In the release notes it shows that the "jackson-databind" library should be 2.18.2,
It seems like the shading in the `shaded.databricks.org.apache.hudi.com` was not done well.
Trying to understand the bug i run the following command
%sh find /databricks/jars -name "*hudi*.jar" -exec jar tf {} \; | grep "META-INF/services/com.fasterxml.jackson"
which returned 'META-INF/services/com.fasterxml.jackson.databind.Module'. this helps me conclude that the bug in a shading issue in the databricks hudi package.
Could someone take a look at this?
Thank you.
