topic Re: How to import a data table from SQLQuery2 into Databricks notebook in Data Engineering

How to import a data table from SQLQuery2 into Databricks notebook

moski — Thu, 22 Dec 2022 14:18:54 GMT

Can anyone show me a few commands to import a table, say "mytable2

From: Microsoft SQL Server

Into: Databricks Notebook using spark dataframe or at least pandas dataframe

Cheers!

Re: How to import a data table from SQLQuery2 into Databricks notebook

irfanaziz — Thu, 22 Dec 2022 21:28:19 GMT

You can read any table from MSSQL. You would need to authenticate to the db, so your would need the connection string:

def dbProps():
 return {
  "user" : "db-user",
  "password" : "your password",
  "driver" : "com.microsoft.sqlserver.jdbc.SQLServerDriver"
 }

So you use the above function to authenticate.

Then read using the below.

url ="jdbc:sqlserver://servername.database.windows.net:1433;database=db-test"
df = spark.read.jdbc(url=url, table="dbo.tablename", properties=dbProps())

Re: How to import a data table from SQLQuery2 into Databricks notebook

Aviral-Bhardwaj — Fri, 23 Dec 2022 03:37:19 GMT

absolutely right but make sure that you have firewall enabled for SQL server

Re: How to import a data table from SQLQuery2 into Databricks notebook

DBXC — Wed, 26 Apr 2023 14:55:07 GMT

Would someone provide example with IntegratedSecurity instead of user name and password?

Is this even possible with Databricks? We do have Azure AD sync'd. Any additional steps to allow IntegratedSecurity in connection string for better security control?