How to import a data table from SQLQuery2 into Databricks notebook
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-22-2022 06:18 AM
Can anyone show me a few commands to import a table, say "mytable2
From: Microsoft SQL Server
Into: Databricks Notebook using spark dataframe or at least pandas dataframe
Cheers!
- Labels:
-
Databricks notebook
-
Table
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-22-2022 01:28 PM
You can read any table from MSSQL. You would need to authenticate to the db, so your would need the connection string:
def dbProps():
return {
"user" : "db-user",
"password" : "your password",
"driver" : "com.microsoft.sqlserver.jdbc.SQLServerDriver"
}So you use the above function to authenticate.
Then read using the below.
url ="jdbc:sqlserver://servername.database.windows.net:1433;database=db-test"
df = spark.read.jdbc(url=url, table="dbo.tablename", properties=dbProps())
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-22-2022 07:37 PM
absolutely right but make sure that you have firewall enabled for SQL server
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-26-2023 07:55 AM
Would someone provide example with IntegratedSecurity instead of user name and password?
Is this even possible with Databricks? We do have Azure AD sync'd. Any additional steps to allow IntegratedSecurity in connection string for better security control?