The agent installations via the init script would install the agents in the Spark containers (All user workloads + spark processes run in the container). The users don't have direct access to the host machine and can't install any agents. You may need to enable cluster log delivery to see if the init script execution logs. Also, try simple commands via notebook job to check the agent status and any other diagnostics related to these agents. Also, the cluster mode may matter here - Shared/User isolation mode may have more restrictions in compared to single user clusters.In case you have cluster mode differing between test cluster and your job cluster.