Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
Showing results for 
Search instead for 
Did you mean: 

Cluster Failed to Start - Cluster scoped init scrip failed: Script exit status is non-zero

New Contributor II

i have been using cluster scoped init script for around 1 year already and everything is working fine. But suddenly, Databricks cluster has failed to restart since last week Thursday (13th June 2024). It returns this error:” Failed to add 2 containers to the compute. Will attempt retry: false. Reason: Init script failure. Cluster scoped init script /EDH_ENIGMA/ failed: Script exit status is non-zero”.


However, for some reason, the script is working fine in PROD. For your information, there is no changes has been done to the cluster or the Init Script for the last 6 months. And we have been using this script and configuration since last year. Even the day before the failure start, the cluster is working fine. So, we want to check in with you if you have any idea why this is happening. Attached below is a few screenshot highlighting this issue:

Here are a few things we have tried:

  1. Delete the script and recreating it again
  2. Change the init script from bin/bash to sh script
  3. Change the extension from .sh to .bash
  4. Configure the script to sudo apt install something to include -y flag


This is the init script that we have been using:



curl --silent | apt-key add -
curl > /etc/apt/sources.list.d/mssql-release.list
# Install msodbcsql17
apt-get update
ACCEPT_EULA=Y apt-get --quiet ---yes install msodbcsql17



Init Script failed in DEV


Same script working in PROD


Here is additonal info about our Databricks Version:

Policy : Unrestricted

Runtime Version : 13.3 LTS (includes Apache Spark 3.4.1, Scala 2.12)


1-2 Workers32-64 GB Memory8-16 Cores
1 Driver32 GB Memory, 8 Cores



Contributor III

How did you recreate the script in DEV? Are you on a Windows machine? Might be worth checking the file's line endings in VSCode, to make sure they are LF (unix line endings). These issues usually occur because of this. I just tested your init script and it works fine for me. 

i create the .sh file in Databricks workspace. Not using any windows machine or VS code for this. The execution was configured through cluster-scoped init script. 

Is the cluster configuration the same? i.e. Shared vs. Non-Isolation shared?
There must be a discrepancy (maybe in cluster permissions) somewhere. 

Valued Contributor III

Just maybe - there is no outbound connection on DEV from Cluster VNET to URL you are trying to get ? 
You can spin ALl purpose cluster and try testing connection with %sh magic command

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!