Need guidance on connecting to Azure Databricks using JDBC Protocol

[subhankar]

Step 1: Download and Reference the JDBC Driver 

Download the Databricks JDBC Driver: 

• Visit the Databricks JDBC Driver download page. 

• Download the appropriate version for your operating system. 

• Extract the DatabricksJDBC42.jar file from the downloaded zip file. 

Step 2: Gather Configuration Settings 

Gather Configuration Settings: 

• Server Hostname: The hostname of your Azure Databricks cluster or SQL warehouse. 

• Port: The port number (default is 443). 

• HTTP Path: The HTTP path to your Databricks SQL endpoint. 

• Authentication Token: A personal access token or Microsoft Entra ID (formerly Azure Active Directory) token. 

Step 3: Create a Personal Access Token 

Create a Personal Access Token: 

• In your Azure Databricks workspace, click your username in the top bar and select Settings. 

• Click Developer. 

• Next to Access tokens, click Manage. 

• Click Generate new token. 

• Optionally, enter a comment and set the token's lifetime. 

• Click Generate and copy the token to a secure location. 

Step 4: Configure the JDBC Connection 

Configure the JDBC Connection: 

• Use the following JDBC URL format: 

jdbc:spark://<Server Hostname>:<Port>/;transportMode=http;ssl=1;httpPath=<HTTP Path>;AuthMech=3;UID=token;PWD=<Personal Access Token> 

• Replace <Server Hostname>, <Port>, <HTTP Path>, and <Personal Access Token> with your specific values. 

Step 5: Connect Using JDBC 

Connect Using JDBC: 

• Depending on your programming language or tool, use the JDBC URL to establish a connection. 

%python 

import jaydebeapi 

jdbc_url = "jdbc:spark://<Server Hostname>:<Port>/;transportMode=http;ssl=1;httpPath=<HTTP Path>;AuthMech=3;UID=token;PWD=<Personal Access Token>" 

driver_class = "com.simba.spark.jdbc.Driver" 

jar_file = "/path/to/DatabricksJDBC42.jar" 

conn = jaydebeapi.connect(driver_class, jdbc_url, jars=jar_file)  

cursor = conn.cursor() 

cursor.execute("SELECT * FROM my_table") 

result = cursor.fetchall() 

cursor.close() 

conn.close() 

I have followed the above steps to connect to Azure Databricks using JDBC protocol. I am using Python to connect but getting below error:

raise JVMNotFoundException("No JVM shared library file ({0}) "
jpype._jvmfinder.JVMNotFoundException: No JVM shared library file (jvm.dll) found. Try setting up the JAVA_HOME environment variable properly.

Please help me with this and also let me know if there is another way without having JVM. Thanks

[Ayushi_Suthar]

Hi @subhankar , Good Day! 

Looking at the error you are getting here shows that it tries to find some kind of JVM file and probably refers to the JAVA_HOME variable to achieve it. It looks as if it is not set correctly in your Environment Variables.

To resolve this issue, you should ensure that Java is correctly installed and configured on your system. Specifically, you should check that the JAVA_HOME environment variable is set to the path of your Java installation directory.

I am also sharing a few public documents which might help you to resolve this issue : https://stackoverflow.com/questions/75656048/this-problem-appeared-with-the-jpyp-library-and-it-give...

https://github.com/iiasa/message_ix/issues/35

Moreover, you can refer to our below document to connect with Databricks using JDBC Drivers: https://docs.databricks.com/en/integrations/jdbc-odbc-bi.html#:~:text=Databricks%20JDBC%20Driver%20G...

Please let me know if this helps and leave a like if this information is useful, followups are appreciated.
Kudos
Ayushi