I'm trying to install Maven Libraries on the job cluster (non interactive cluster) as part of databricks workflow. I've added the context in the cluster configuration as part of deployment which I cant find the same in the post deployment configuration. Below is the config for reference.
Any thoughts highly appreciated.
I found the workaround. Below are the steps:
1. Add the required library to the Allowed list at the workspace level (require workspace/metastore admin access); you might need coordinates groupdd:artifactId:version
2. At the task level, include under Dependent libraries
This approach will install the 3rd party libraries without additional configuration at job compute.
