topic Re: Error while installing ODBC to shared cluster in Data Engineering

Error while installing ODBC to shared cluster

Poovarasan — Tue, 02 Jul 2024 12:06:05 GMT

I previously used the following script to install and configure the ODBC driver on our shared cluster in Databricks, and it was functioning correctly. However, I am currently experiencing issues where the installation is not working as expected. Please let us know if anyone has solution.

#!/bin/bash
curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
curl https://packages.microsoft.com/config/ubuntu/$(lsb_release -rs)/prod.list > /etc/apt/sources.list.d/mssql-release.list
sudo apt-get update
sudo ACCEPT_EULA=Y apt-get -q -y install msodbcsql17
sudo apt-get -y install unixodbc-dev

Re: Error while installing ODBC to shared cluster

Witold — Tue, 02 Jul 2024 12:19:58 GMT

Could you share the error message with us?

Re: Error while installing ODBC to shared cluster

daniel_sahal — Tue, 02 Jul 2024 12:21:06 GMT

@Poovarasan I need more info here. Why it's not working? What kind of error message is it showing? What are you environment details (DBR, access mode etc.)?

Re: Error while installing ODBC to shared cluster

Poovarasan — Tue, 02 Jul 2024 12:25:51 GMT

Hi @Witold & @daniel_sahal

Utilized Databricks Runtime (DBR): 14.3 LTS, which includes Apache Spark 3.5.0 and Scala 2.12. and please see below attached screenshot for more details.

DBR : 14.3 LTS (includes Apache Spark 3.5.0, Scala 2.12)

Re: Error while installing ODBC to shared cluster

Witold — Tue, 02 Jul 2024 12:38:51 GMT

I honestly doubt that it worked previously, as this isn't supported on shared cluster: Check this and this for further information. I hope it helps!

Re: Error while installing ODBC to shared cluster

Poovarasan — Wed, 03 Jul 2024 07:17:46 GMT

Hi @Witold ,

I attempted to use a cluster-scoped init script, but it resulted in the following error:

INVALID_PARAMETER_VALUE: Attempting to install the following init scripts that are not in the allowlist. /Volumes/odbc/init_install_sqlserver_odbc (1).sh: PERMISSION_DENIED: '/Volumes/odbc/init_install_sqlserver_odbc (1).sh' is not in the artifact allowlist.

Re: Error while installing ODBC to shared cluster

Witold — Wed, 03 Jul 2024 07:27:52 GMT

is not in the artifact allowlist

Did you follow these docs?

Re: Error while installing ODBC to shared cluster

imsabarinath — Wed, 17 Jul 2024 17:05:09 GMT

The below approach is working for me... I had to download the packages upfront and place it on a volume though.

#!/bin/bash
set -euxo pipefail
echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
sudo ACCEPT_EULA=Y dpkg -i odbcinst_2.3.9-5_amd64.deb
sudo ACCEPT_EULA=Y dpkg -i msodbcsql17_17.10.1.1-1_amd64.deb
sudo ACCEPT_EULA=Y dpkg -i libsasl2-modules-gssapi-mit_2.1.27+dfsg2-3ubuntu1.2_amd64.deb