How to install a JAR library via a global init script?

[ncouture]

I have a JAR I want to be installed as a library on all clusters. I have tried both

wget /databricks/jars/ some_repo

and

cp /dbfs/FileStore/jars/name_of_jar.jar /databricks/jars/

clusters start up but the JAR is not installed as a library. I am aware that it might not show in the UI under libraries however its effect allows the querying of JSON formatted tables. Testing with a newly started/created table results in an error when querying such a table. The only way I have gotten this to work is through the GUI on a cluster by cluster basis using the path in the second snippet. So the file is good.

What am I missing to doing this programmatically via an init script?

[karthik_p]

@Nicholas Couture​ please try below steps in article, if you have not come across, you need to go with global init config please. Cluster node initialization scripts | Databricks on AWS

[ncouture]

Yes I have viewed that article already. It contains nothing new or helpful.

I can see the JAR in the system classpath of the Spark UI but it doesn't seem to be installed/initialised for the cluster. This is really quite annoying

[ncouture · Accepted solution]

Found a solution.

echo /databricks/databricks-hive /databricks/jars /databricks/glue | xargs -n 1 cp /dbfs/FileStore/jars/NAME_OF_THE_JAR.jar

had to first add the jar as a library through the GUI via Create -> Library then uploaded the downloaded JAR. Copying it into the three locations noted sorted it out