cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Run result unavailable: job failed with error message Library installation failed for library due to user error for jar: "dbfs:/my-jar.jar"

mani238
New Contributor III

Run result unavailable: job failed with error message Library installation failed for library due to user error for jar: "dbfs:/my-jar.jar" . Error messages: Library installation attempted on the driver node of cluster 0510-013936-3cc6d9kw and failed. Please refer to the following error message to fix the library or contact Databricks support. Error Code: DRIVER_LIBRARY_INSTALLATION_FAILURE. Error Message: java.util.concurrent.ExecutionException: java.io.FileNotFoundException: dbfs:/my-jar.jar

How do i download and install the " jar: "dbfs:/my-jar.jar" files

1 ACCEPTED SOLUTION

Accepted Solutions

Atanu
Esteemed Contributor
Esteemed Contributor

@manivannan p​ either you need to install it manually from library option of interactive cluster https://docs.databricks.com/libraries/workspace-libraries.html#upload-a-jar-python-egg-or-python-whe...

or from library option of job configuration -

JAR: Specify the Main class. Use the fully qualified name of the class containing the main method, for example, 

org.apache.spark.examples.SparkPi

. Then click Add under Dependent Libraries to add libraries required to run the task. One of these libraries must contain the main class.

https://docs.databricks.com/data-engineering/jobs/jobs.html#create-a-job

But from the error message, this looks like the jar file is invalid or there might be some issue with the jar build.

View solution in original post

3 REPLIES 3

Kaniz
Community Manager
Community Manager

Hi @manivannan p​ , Please refer to this S.O thread.

Also,

Problem

A Databricks job fails because the job requires a library that is not yet installed, causing Import errors.

Cause

The error occurs because the job starts running before the required libraries are installed.

If you run a job on a cluster in either of the following situations, the cluster can experience a delay in installing libraries:

  • When you start an existing cluster with libraries in the terminated state.
  • When you start a new cluster that uses a shared library (installed on all clusters).

Solution

Atanu
Esteemed Contributor
Esteemed Contributor

@manivannan p​ either you need to install it manually from library option of interactive cluster https://docs.databricks.com/libraries/workspace-libraries.html#upload-a-jar-python-egg-or-python-whe...

or from library option of job configuration -

JAR: Specify the Main class. Use the fully qualified name of the class containing the main method, for example, 

org.apache.spark.examples.SparkPi

. Then click Add under Dependent Libraries to add libraries required to run the task. One of these libraries must contain the main class.

https://docs.databricks.com/data-engineering/jobs/jobs.html#create-a-job

But from the error message, this looks like the jar file is invalid or there might be some issue with the jar build.

Anonymous
Not applicable

Hey there @manivannan p​ 

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. 

We'd love to hear from you.

Thanks!

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.