jacob_hill_prof
New Contributor II

@Gleb Smolnik​ You might also want to try cloning a github repo in your init script and then storing dependencies like requirements.txt files and other init scripts there. By doing this you can pull a whole slew of init scripts to be utilized in your cluster dynamically from a versioned source.

init.sh

git clone <github repo url>my_repo.git
git -C ./my_repo checkout common_cluster_init # checkout non-main branches
pip install -r ./my_repo/init/dbricks_clusters/requirements.txt # use scripts in the repo