Jackson version conflict

Shimon
New Contributor II

Hi,

I am trying to implement the Spark TableProvider api and i am experiencing a jar conflict (I am using the 17.3 runtime).

 

com.fasterxml.jackson.databind.JsonMappingException: Scala module 2.15.2 requires Jackson Databind version >= 2.15.0 and < 2.16.0 - Found jackson-databind version 2.18.2
	at shaded.databricks.org.apache.hudi.com.fasterxml.jackson.module.scala.JacksonModule.setupModule(JacksonModule.scala:61)
	at shaded.databricks.org.apache.hudi.com.fasterxml.jackson.module.scala.JacksonModule.setupModule$(JacksonModule.scala:46)
	at shaded.databricks.org.apache.hudi.com.fasterxml.jackson.module.scala.DefaultScalaModule.setupModule(DefaultScalaModule.scala:17)
	at com.fasterxml.jackson.databind.ObjectMapper.registerModule(ObjectMapper.java:908)
	at com.fasterxml.jackson.databind.ObjectMapper.registerModules(ObjectMapper.java:1110)
	at com.fasterxml.jackson.databind.ObjectMapper.findAndRegisterModules(ObjectMapper.java:1194)

 

In the release notes it shows that the "jackson-databind" library should be 2.18.2,
It seems like the shading in the `shaded.databricks.org.apache.hudi.com` was not done well.
Trying to understand the bug i run the following command

 

 

%sh find /databricks/jars -name "*hudi*.jar" -exec jar tf {} \; | grep "META-INF/services/com.fasterxml.jackson"

 

 

which returned 'META-INF/services/com.fasterxml.jackson.databind.Module'. this helps me conclude that the bug in a shading issue in the databricks hudi package.

Could someone take a look at this?
Thank you.