@Juned Mala​ :

To access the /Workspace/Shared/ directory in a shell script, you can use the Databricks CLI command databricks fs to navigate the Databricks File System (DBFS).

Here's an example of how to install a library from /Workspace/Shared/ using the init.sh script.

#!/bin/bash
 
# Install library from /Workspace/Shared/
databricks fs cp dbfs:/Workspace/Shared/my_library-1.0.jar /tmp/my_library.jar
databricks libraries install --cluster-id <cluster-id> --jar /tmp/my_library.jar
 
# Add other cluster initialization steps here

In this example, we use databricks fs cp to copy the library from /Workspace/Shared/ to /tmp/ in the DBFS. Then we use databricks libraries install to install the library on the cluster, specifying the path to the library in /tmp/.

Note that you'll need to replace <cluster-id> with the ID of your Databricks cluster. You can find the cluster ID by navigating to the cluster in the Databricks workspace and looking at the URL, which will contain a string like clusterId=1234-567890-abcdefg.