topic Re: Install ODBC driver by init script in Data Engineering

Install ODBC driver by init script

LukaszJ — Mon, 04 Apr 2022 12:43:49 GMT

Hello,

I want to install ODBC driver (for pyodbc).

I have tried to do it using terraform, however I think it is impossible.

So I want to do it with Init Script in my cluster. I have the code from the internet and it works when it is on the beginning of the cluster:

curl https://packages.microsoft.com/keys/microsoft.asc | apth-key add -
curl https://packages.microsoft.com/config/ubuntu/16.04/prod.list > /etc/apt/sources.list.d/mssql-release/list
apt-get update
ACCEPT_EULA=Y apt-get install msodbcsql17
apt-get -y install unixodbc-dev

So I make a init script:

file_path = "/databricks/init_script/my_scipy.bash"
file_content = """
curl https://packages.microsoft.com/keys/microsoft.asc | apth-key add -
curl https://packages.microsoft.com/config/ubuntu/16.04/prod.list > /etc/apt/sources.list.d/mssql-release/list"""
apt-get update
ACCEPT_EULA=Y apt-get install msodbcsql17
apt-get -y install unixodbc-dev"""
dbutils.fs.put(file_path, file_contnet, True)

And the problem is in 5 line (apt-get update).

Without the line driver does not works.

With the line cluster cannot start running because: Script exit status is non-zero

Do you know what should I do?

Best regards,

Łukasz

Re: Install ODBC driver by init script

Hubert-Dudek — Mon, 04 Apr 2022 12:50:37 GMT

This is code that I am using for pyodbc init script + also in cluster/tasks settings I have added PyPI

pyodbc library.

dbutils.fs.put("/databricks/scripts/pyodbc-install.sh","""
#!/bin/bash
sudo apt-key add /dbfs/databricks/scripts/microsoft.asc
sudo cp -f /dbfs/databricks/scripts/prod.list /etc/apt/sources.list.d/mssql-release.list
sudo apt-get update
sudo ACCEPT_EULA=Y apt-get install msodbcsql17""", True)

Re: Install ODBC driver by init script

Hubert-Dudek — Mon, 04 Apr 2022 12:53:37 GMT

Plus I downloaded permanently Microsoft libraries as they indeed crushed the server (often that page even haven't worked)

%sh
sudo curl -k https://packages.microsoft.com/keys/microsoft.asc > /dbfs/databricks/scripts/microsoft.asc 
sudo curl -k https://packages.microsoft.com/config/ubuntu/16.04/prod.list > /dbfs/databricks/scripts/prod.list

Re: Install ODBC driver by init script

LukaszJ — Mon, 04 Apr 2022 13:53:04 GMT

Thank you Hubert for help.

Your code works for me! 😊

Best regards,

Łukasz

Re: Install ODBC driver by init script

Shourya — Thu, 26 Jan 2023 20:36:26 GMT

Could you please share how you solved the problem? I'm facing the same issue

Re: Install ODBC driver by init script

MayaBakh_80151 — Tue, 29 Aug 2023 18:27:53 GMT

Is there a way to install via a notebook only without using dbfs init script?
With new DBR policy of no init script. I **bleep** need to migrate shell script solution to a notebook on my workspace.
any leads where can we start?

Re: Install ODBC driver by init script

MayaBakh_80151 — Tue, 29 Aug 2023 21:17:43 GMT

Actually found this article and using this to migrate my shell script to workspace.
Cluster-named and cluster-scoped init script migration notebook - Databricks

Re: Install ODBC driver by init script

GowthamR — Wed, 17 Sep 2025 12:29:20 GMT

Hi Hubert,

Good Day!

Iam also trying to connect to sql server from Databricks via pyodbc, where I need to add the init script in cluster configuration, However what iam doing now is, having the above mentioned code in the text file and saving with .sh extension,and uploaded the file in s3 , and using that path in the init script file addition, but however it is returning me "Script Exit Status as Non Zero", So may I what can be done?