You don't need an ODBC driver inside a Databricks job cluster to run stored procedures, and init scripts are fragile enough that they easily break cluster startup.

Options:

Option 1 (Recommended): Use built-in JDBC / connectors instead of ODBC

For SQL Server and most RDBMS, Databricks already ships JDBC-based connectors; you can call stored procedures from notebooks without installing any OS-level driver. For example, SQL Server:

driver = "com.microsoft.sqlserver.jdbc.SQLServerDriver"
host = "<server-host>"
port = "1433"
database = "<db-name>"
user = "<user>"
password = "<password>"

url = f"jdbc:sqlserver://{host}:{port};database={database}"

df = (spark.read
      .format("jdbc")
      .option("driver", driver)
      .option("url", url)
      .option("dbtable", "dbo.SomeTable")  # Or a view wrapping a stored proc
      .option("user", user)
      .option("password", password)
      .load())

On newer runtimes you can also use the sqlserver data source or Lakehouse Federation / remote_query to run native SQL (including EXEC MyProc …) directly against SQL Server, again with no init script.
Recommendation: If your goal is “run stored procedures from a notebook”, switch to these JDBC/federation options instead of ODBC; they’re supported, simpler, and far less brittle.

Option 2: Template cluster-scoped init script to install an ODBC driver

If you truly must use ODBC (for example, a tool library that only supports pyodbc), use a cluster-scoped init script stored in a Workspace file or UC Volume (DBFS init scripts are end-of-life and can prevent clusters from starting).

Example install_odbc.sh (Ubuntu-style driver, adjust URL & bits for your vendor):

#!/bin/bash
set -euxo pipefail

LOG_DIR=/tmp/odbc-init
mkdir -p "$LOG_DIR"

echo "[$(date)] Starting ODBC init script" >> "$LOG_DIR/odbc-init.log"

# 1. System prerequisites
apt-get update
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
  unixodbc unixodbc-dev curl

# 2. Download & install your vendor ODBC driver (.deb example)
DRIVER_URL="https://<vendor>/path/to/driver_amd64.deb"
DRIVER_DEB="/tmp/driver.deb"

curl -fsSL "$DRIVER_URL" -o "$DRIVER_DEB"
dpkg -i "$DRIVER_DEB" || DEBIAN_FRONTEND=noninteractive apt-get install -f -y

# 3. (Optional) Register driver/DSN
# cat >> /etc/odbcinst.ini <<'EOF'
# [MyVendorDriver]
# Description=My DB ODBC Driver
# Driver=/opt/myvendor/lib/libmyodbc.so
# EOF
#
# cat >> /etc/odbc.ini <<'EOF'
# [MyVendorDSN]
# Driver=MyVendorDriver
# Server=<server>
# Database=<database>
# Port=<port>
# EOF

echo "[$(date)] Finished ODBC init script" >> "$LOG_DIR/odbc-init.log"
exit 0

Key points so the cluster keeps starting:

• Store the script at a supported path (for example /Workspace/.../install_odbc.sh or /Volumes/<cat>/<sch>/<vol>/install_odbc.sh) and configure it as a cluster-scoped init script on the job’s compute.
• Keep it small (<64 KB) and make sure it always exits 0 on success; any non-zero code causes the job cluster to fail to start.
• On Standard access mode clusters, the script path must be allowlisted by a metastore admin, or it will be blocked.
• If it “stops working after some time”, the usual culprits are:
  • Script stored on DBFS root (now disallowed)
  • Script path no longer on the allowlist after moving to Standard mode
  • Upstream package URL changed and curl/apt-get now fails (check init logs)

You can inspect failures in /databricks/init_scripts/...stdout.log or via cluster log delivery.

Option 3: Use Lakehouse Federation / remote_query to execute procedures

For SQL Server and several other databases, you can create a Unity Catalog connection and then use remote_query to run the database’s native SQL (including stored procedures) from Databricks SQL or notebooks, again without any OS-level driver:

SELECT *
FROM remote_query(
  'my_sqlserver_connection',
  database => 'MyDb',
  query    => 'EXEC dbo.MyStoredProc @Param1 = 42'
);

This uses JDBC under the hood and is the strategic path for cross-system querying.

My recommendation:
Use Option 1 or 3 (JDBC / Lakehouse Federation / remote_query) instead of installing an ODBC driver on the job cluster. Only use Option 2 if you’re blocked by a hard dependency on ODBC, and in that case, adapt the template script above and double-check script location, allowlist, and init-script logs to understand why it started failing.