cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

What's the recommended way to use secrets within Global init scripts?

Anonymous
Not applicable
 
3 REPLIES 3

User16826994223
Honored Contributor III

The order of execution of init scripts is:

  1. Legacy global (deprecated)
  2. Cluster-named (deprecated)
  3. Global (new)
  4. Cluster-scoped

Arfrenece link that can help is - https://docs.databricks.com/clusters/init-scripts.html#global-init-scripts

sajith_appukutt
Honored Contributor II

You could specify a secret path in an environment at the cluster level

SPARKPASSWORD=<path-value>

and this would be available in global init scripts / cluster-scoped init script

if [[ $SPARKPASSWORD ]]; then
  use $SPARKPASSWORD
fi

sstrong99
New Contributor II

I also have this question, and I don't understand the answer from @Sajith Appukuttanโ€‹ . This solution would require configuring each new cluster separately with the correct environment variable, correct? That seems like it defeats the purpose of using a global init script. Is there a method that does not require configuration of each new cluster separately?

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.