- 3397 Views
- 2 replies
- 1 kudos
I specify init scripts in my deployment.conf, as below: basic-static-cluster: &basic-static-cluster
new_cluster:
spark_version: "13.0.x-scala2.12"
num_workers: 1
node_type_id: "n2-highmem-2"
init_scripts:
- worksp...
- 3397 Views
- 2 replies
- 1 kudos
Latest Reply
Hi @Naga Vaibhav Elluru We haven't heard from you since the last response from @Debayan Mukherjee , and I was checking back to see if her suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be he...
1 More Replies
- 8208 Views
- 8 replies
- 4 kudos
I am trying to run a cluster-scoped init script through Pulumi. I have referred to this documentation https://learn.microsoft.com/en-us/azure/databricks/clusters/configure#spark-configuration However, looks like the documentation is not very clear.I ...
- 8208 Views
- 8 replies
- 4 kudos
Latest Reply
Hi @Sulfikkar Basheer Shylaja , Why don't you store the init-script on DBFS and just pass the dbfs:/ path of the init script in Pulumi? You could just run this code on a notebook-%python
dbutils.fs.put("/databricks/init-scripts/set-private-pip-repos...
7 More Replies
- 4341 Views
- 4 replies
- 8 kudos
When I run the init_script during cluster creationapt-get update && apt-get install -y ffmpeg libsndfile1-devI get an error in cluster logs E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/universe/o/openal-soft/libopenal1_1.19.1-1_amd64.deb ...
- 4341 Views
- 4 replies
- 8 kudos
Latest Reply
Hi @Sergii Ivakhno Could you please check if outbound TCP access for port 80 is allowed in the security group.
3 More Replies
- 5828 Views
- 7 replies
- 4 kudos
Hi there, if I set any secret in an env var to be used by a cluster-scoped init script, it remains available for the users attaching any notebook to the cluster and easily extracted with a print.There's some hint in the documentation about the secret...
- 5828 Views
- 7 replies
- 4 kudos
Latest Reply
@Fermin Vicente good to know that this approach is working well. but please make sure that you use this approach at the end of your init script only
6 More Replies
