topic Re: Assistance Required: Integrating Databricks ODBC Connector with Azure App Service in Administration & Architecture

Assistance Required: Integrating Databricks ODBC Connector with Azure App Service

nanda_ — Tue, 31 Dec 2024 12:17:38 GMT

Hi,

I have successfully established an ODBC connection with Databricks to retrieve data from the Unity Catalog in a local C# application using the Simba Spark ODBC Driver, and it is working as expected.

I now need to integrate this functionality into an existing .NET C# application hosted on an Azure App Service (Linux). However, I have not been able to locate any documentation or resources specific to implementing the ODBC connector in this environment.

Could you kindly provide any relevant documentation, guidelines, or suggestions on how to achieve this integration? Your assistance would be greatly appreciated.

using System;
using System.Data.Odbc;

class Program
{
static void Main()
{
// Connection details

// ODBC connection string
//string connectionString = $"Driver={{Simba Spark ODBC Driver}};" +
// $"Host={serverHostname};" +
// $"HTTPPath={httpPath};" +
// $"AuthMech=3;" +
// $"UID=token;" +
// $"PWD={accessToken};";

try
{
// Connect to the database
using (OdbcConnection connection = new OdbcConnection(connectionString))
{
connection.Open();
Console.WriteLine("Connected to Databricks!");

// Query to execute
string query = "SELECT * FROM Table LIMIT 5";

using (OdbcCommand command = new OdbcCommand(query, connection))
{
using (OdbcDataReader reader = command.ExecuteReader())
{
Console.WriteLine("Query Results:");
while (reader.Read())
{
Console.WriteLine(reader["column"]);
}
}
}
}
}
catch (Exception ex)
{
Console.WriteLine("Error: " + ex.Message);
}
}
}

Re: Assistance Required: Integrating Databricks ODBC Connector with Azure App Service

Walter_C — Tue, 31 Dec 2024 12:32:23 GMT

To integrate the Simba Spark ODBC Driver into your existing .NET C# application hosted on an Azure App Service (Linux), you can follow these steps:

Install the ODBC Driver on Azure App Service (Linux):
- Download the Simba Spark ODBC Driver for Linux from the Databricks website.
- Use the RPM or tarball package to install the driver on your Azure App Service. For example, if using the tarball package, you can extract it to the `/opt` directory:
```
tar --directory=/opt -zxvf SimbaSparkODBC-[Version].[Release]-Linux.tar.gz
```

Configure the ODBC Driver:

Set up the odbc.ini and odbcinst.ini configuration files. These files should be placed in a directory accessible by the application, such as /etc.

Example `odbcinst.ini` configuration:

[ODBC Drivers]
Simba Apache Spark ODBC Connector=Installed

[Simba Apache Spark ODBC Connector]
Description=Simba Apache Spark ODBC Connector
Driver=/opt/simba/spark/lib/64/libsparkodbc_sb64.so

Example odbc.ini configuration:

[ODBC Data Sources]
Sample DSN=Simba Apache Spark ODBC Connector

[Sample DSN]
Driver=/opt/simba/spark/lib/64/libsparkodbc_sb64.so
Host=your-databricks-host
Port=443
AuthMech=3
UID=token
PWD=your-access-token

Set Environment Variables:
- Ensure the environment variables are set to point to the ODBC configuration files. Add the following to your startup script or environment configuration:
```
export ODBCINI=/etc/odbc.ini
export ODBCINSTINI=/etc/odbcinst.ini
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/simba/spark/lib/64
```
Modify Your C# Application:
- Update your connection string in the C# application to use the DSN configured in the odbc.ini file:
```
  string connectionString = "DSN=Sample DSN;";
```

Re: Assistance Required: Integrating Databricks ODBC Connector with Azure App Service

szymon_dybczak — Tue, 31 Dec 2024 12:36:31 GMT

Hi @nanda_ ,

So basically what you need to do is to install simba odbc driver on your Azure App Service environment. Then your code should work in the same way as in your local machine.

One possibility is to use Windows or Linux Containers on Azure App Service in which you have control over what drivers or custom software to install.

In Dockerfile you can installl databricks ODBC driver in following way (adjust driver version to your needs) :

# Install Databricks ODBC driver. RUN apt update && apt install -y unixodbc unixodbc-dev freetds-dev sqsh tdsodbc unzip libsasl2-modules-gssapi-mit RUN curl -sL https://databricks.com/wp-content/uploads/drivers-2020/SimbaSparkODBC-2.6.16.1019-Debian-64bit.zip -o databricksOdbc.zip && unzip databricksOdbc.zip RUN dpkg -i SimbaSparkODBC-2.6.16.1019-Debian-64bit/simbaspark_2.6.16.1019-2_amd64.deb RUN export ODBCINI=/etc/odbc.ini ODBCSYSINI=/etc/odbcinst.ini SIMBASPARKINI=/opt/simba/spark/lib/64/simba.sparkodbc.ini

If you're using the source code deployment option, then you can try to install Simba ODBC driver using SSH.