cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Pass secret in spark config when value is in form a.b.c={{secrets/scope/secret}}

macmiller1
New Contributor II

I am configuring the Cluster for a spark-submit task and I am trying to specify `spark.executor.extraJavaOptions a.b.c={{secrets/scope/secret}}` but the literal {{secrets/scope/secret}} is being passed in rather than the secret value itself.

I know the convention for secrets is `spark.conf.something {{secrets/scope/secret}}` when the value is the secret itself and this substitution works fine for me in similar cases. Given my value in the above is a.b.c={{secrets/scope/secret}} the secret substitution is not working. I have also tried with an environment variable ie `spark.executor.extraJavaOptions a.b.c=${MY_ENV_VAR}` but this is also not working. 

I suspect that given the {{secrets/scope/secret}} does not have a space character before it but instead `=` the substitution isn't working properly.

Any suggestions on how to achieve this would be greatly appreciated!

 

0 REPLIES 0

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now