- 3592 Views
- 4 replies
- 4 kudos
Make environment variables defined in init script available to Spark JVM job?
Hi,We're using Databricks Runtime version 11.3LTS and executing a Spark Java Job using a Job Cluster. To automate the execution of this job, we need to define (source in from bash config files) some environment variables through an init script (clust...
- 3592 Views
- 4 replies
- 4 kudos
- 4 kudos
Hi @Rahul K​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!
- 4 kudos
- 26858 Views
- 4 replies
- 2 kudos
Resolved! Set environment variables in global init scripts
Hi Databricks Community,I want to set environment variables for all clusters in my workspace.The goal is to have environment (dev, prod) specific environment variables values.Instead of set the environment variables for each cluster, a global script ...
- 26858 Views
- 4 replies
- 2 kudos
- 2 kudos
We have set the env variable at Global Init script as below,sudo echo DATAENV=DEV >> /etc/environmentand we try to access the variable in notebook that run with "Shared" cluster mode. import os print(os.getenv("DATAENV"))But the env variable is not a...
- 2 kudos