Databricks

uberweiss · ‎08-31-2023

We have previously been able to access our Databricks cluster in R using ODBC but it stopped working a couple of months ago and now i can't get it to connect.

I've downloaded the latest drivers and added the right information in odbc/odbcinst files but i still get an error when i try to connect.

Below is the code i'm running and the error i get:

conn <- odbcConnect("Databricks")Warning messages:
1: In RODBC::odbcDriverConnect("DSN=Databricks") :
  [RODBC] ERROR: state HY000, code 1170, message [unixODBC][Simba][DriverSupport] (1170) Unexpected response received from server. Please ensure the server host and port specified for the connection are correct.
2: In RODBC::odbcDriverConnect("DSN=Databricks") : ODBC connection failed

And here's the details from my odbc.ini file:

[ODBC Data Sources]
Databricks=Databricks ODBC Connector

[Databricks]
Driver=Simba Spark ODBC Driver 64-bit
Server=our_server.com
HOST=our_server.com
PORT=443
SparkServerType=3
ThriftTransport=2
SSL=1
AuthMech=3
UID=token
PWD=my_token

And below is from the odbcinst file:

[Simba Spark ODBC Driver 64-bit]
Description=Simba Spark ODBC Driver (64-bit)
Driver=/opt/simba/spark/lib/64/libsparkodbc_sb64.so

We have no issues on the same server to access the cluster from Python using the databricks-sql-connector so there should not be any issues with connectivity.

Any suggestions?

Kaniz · ‎08-31-2023

Hi @uberweiss, Based on the error message you are receiving, the issue might be related to the server host or port specified for the connection. Here are a few suggestions to troubleshoot this issue.

1. Verify Server Details: The error message suggests that there might be an issue with the server host and port specified for the connection. Double-check these details in your odbc.ini file against the details provided in the Databricks workspace.

You can retrieve the connection details for your cluster by following these steps:
  - Log in to your workspace.
  - In the sidebar, click **Compute**.
  - Choose a cluster to connect to.
  - Navigate to **Advanced Options**.
  - Click on the **JDBC/ODBC** tab.
  - Copy the connection details

2. Check Permissions: Ensure you have the necessary permissions to connect to the running cluster. You need "Can Attach To" permission to connect to the running cluster and "Can Restart" permission to automatically trigger the cluster to start if its state is terminated when connecting.

3. Verify Authentication: The Databricks platform supports authentication by using a personal access token. Make sure that your personal access token is correct and has not expired.

4. Test Connectivity: Since you mentioned that you could access the cluster from Python using the databricks-sql-connector, try to use the same server details to connect from R.

If none of these suggestions solves the issue, it might be helpful to contact Databricks support by filing a support ticket for further assistance.

Sources:
- [Docs: jdbc-odbc-bi](https://docs.databricks.com/integrations/jdbc-odbc-bi.html)
- [Docs: databricks-connect-legacy](https://docs.databricks.com/dev-tools/databricks-connect-legacy.html)
- [Docs: rstudio](https://docs.databricks.com/sparkr/rstudio.html)