How to update a SQL Server Table using JDBC or something else in Python/Pyspark ?

berserkersap
Contributor

‎06-29-2023 02:58 AM

I need to update a SQL Server Table from Databricks notebook. Right now, I am trying to do this using JDBC. However, it seems we can only append or overwrite the table using the JDBC Connection.

Query databases using JDBC - Azure Databricks | Microsoft Learn

I wanted to try pyodbc and used "pip install pyodbc"

but when I tried to connect

 

pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER='+server+';DATABASE='+database+';ENCRYPT=yes;UID='+username+';PWD='+ password)

 

I got the error message

berserkersap_0-1688032497010.png

('01000', "[01000] [unixODBC][Driver Manager]Can't open lib 'ODBC Driver 17 for SQL Server' : file not found (0) (SQLDriverConnect)")

Please suggest me any way to do this.

Please note that I do not want to use Scala. Also, I also do not want to install a driver or something of that sort on the cluster.

If installing is inevitable, please list down the steps I need to do for that along with the necessary precautions.

0 Kudos