I am trying to run a cluster-scoped init script through Pulumi. I have referred to this documentation

https://learn.microsoft.com/en-us/azure/databricks/clusters/configure#spark-configuration

However, looks like the documentation is not very clear.

I have an init script called set-private-pip-repositories-relay-feed-init-script.sh with the below content.

#!/bin/bash
if [-n  $DEVOPS_GIT_PAT]; then
    use $DEVOPS_GIT_PAT
else
    echo "Couldn't fetch PAT token"
fi
printf "[global]\n" > /etc/pip.conf
printf "extra-index-url=" >> /etc/pip.conf
printf "https://$DEVOPS_GIT_PAT@artificat_url/pypi/simple/\n" >> /etc/pip.conf
echo "script executed"

I tried to add the init scripts in the cluster configuration of Pulumi as below

init_scripts=[
 
    db.ClusterInitScriptArgs(
 
        file=db.ClusterInitScriptFileArgs(
 
            destination="./files/set-private-pip-repositories-relay-feed-init-script.sh"
 
        )
 
    )
 
],

Where should I keep the Databricks secret scope? is it in spark_env_vars={"DEVOPS_GIT_PAT": "{{secrets/azureScope/devopsGitPat}}"} ?

Only a few points are mentioned on this subject, so looking forward to getting advice from someone 🙂