load data from sql server : python works well but spark does not

Data Engineering

Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.

Hi, i have a problem with my on-premise sql conection from databricks.

i have my python code wich use pymssql and it is working well but my spark code doest not and i am using the same credential for both, my spark code is

"""

# Read data from SQL Server into DataFrame

df = spark.read \

.format("jdbc") \

.option("url", f"jdbc:sqlserver://{host}:1433;database={db};user={user};password={password}") \

.option("dbtable", "{table}") \

.load()

"""

i am using the 9.1 LTS (includes Apache Spark 3.1.2, Scala 2.12) and the com.microsoft.azure:spark-mssql-connector_2.12:1.2.0 connector which are compatible.

the sql autentication is mixed

the return error is "com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user"

i dont know what could happen because the connector and the version of spark are compatible and the python code works well so the credential are fine, what else could be going on?

thanks