cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Issue with Maven Dependency Resolution in Databricks (json-smart)

VictorFerron
New Contributor III

Hi,
I'm experiencing issues installing a library in my Databricks cluster due to a Maven dependency resolution error. Specifically, when trying to install or use libraries that depend on net.minidev:json-smart, I get the following error:

 

 
Library installation attempted on the driver node and failed. Error code: ERROR_MAVEN_LIBRARY_RESOLUTION, error message: Library resolution failed because unresolved dependency: net.minidev:json-smart:[1.3.1,2.3]: not found. 
 
After researching, it seems that there is an issue with the Maven metadata for json-smart, causing some versions to be unresolved. I've tried:
 
  • Manually specifying json-smart:2.4.8 in the cluster library settings.
  • Adding alternative repositories (https://repo1.maven.org/maven2/, https://jitpack.io).
  • Cleaning the cluster library cache and restarting.
  • Excluding json-smart from Azure SDK dependencies to force a specific version.

None of these have resolved the issue. Has anyone encountered this problem in Databricks? Any suggestions on how to force a specific version or work around the Maven metadata issue?

Thanks in advance!

1 ACCEPTED SOLUTION

Accepted Solutions

VictorFerron
New Contributor III

To avoid this issue, I manually excluded the json-smart dependency when installing affected libraries and forced a stable version instead.
If you're facing this issue, this could be a viable solution until the json-smart team fixes the metadata in Maven Central.

View solution in original post

4 REPLIES 4

MLob
New Contributor II

Same problem, same tests, same results. Currently working on it, support is very appreciated!

VictorFerron
New Contributor III

To avoid this issue, I manually excluded the json-smart dependency when installing affected libraries and forced a stable version instead.
If you're facing this issue, this could be a viable solution until the json-smart team fixes the metadata in Maven Central.

MLob
New Contributor II

Thanks VitorFerron, exlcuding json-smart dependency worked for installing affected library with Maven (azure-eventhubs-spark_2.12:2.3.22 in my case).

When you say force a stable version instead, do you mean explicitly install json-smart library on cluster with Maven (in my case net.minidev:json-smart:2.3 for example)?


VictorFerron
New Contributor III

You're welcome

Yes, by ‘stable version,’ I mean using a Spark connector version that allows excluding json-smart. Installing an older version manually (like 2.3) may still cause issues, so if excluding it works, I’d recommend sticking with that approach.

Captura de pantalla 2025-02-12 151739.png

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group