Connecting to Synapse database using AzureCliCredential token in Spark

PGrover — Tue, 06 Jun 2023 20:30:15 GMT

I want to connect to my Azure Synapse database using Spark. I can do this in pyodbc no problem but that is not what I want.

Here is how I get my credentials

credential = AzureCliCredential()
databaseToken = credential.get_token('https://database.windows.net/')
 
# get bytes from token obtained
tokenb = bytes(databaseToken[0], "UTF-8")
exptoken = b''
for i in tokenb:
 exptoken += bytes({i})
 exptoken += bytes(1)
tokenstruct = struct.pack("=i", len(exptoken)) + exptoken

I tried this to see if it would work but I am obviously not doing this correctly.

cambio_data_lake = '<synapse database>.sql.azuresynapse.net'
SQL_COPT_SS_ACCESS_TOKEN = '1256'
 
gauges_table = (spark.read
  .format("sqlserver")
  .option("host", cambio_data_lake)
  .option("port", "3342") # optional, can use default port 1433 if omitted
  .option("dbtable", "cambioinspection")
  .option(SQL_COPT_SS_ACCESS_TOKEN,tokenstruct)
  .load()
)

Is there another approach for connecting using a token?

Thanks!

Re: Connecting to Synapse database using AzureCliCredential token in Spark

Anonymous — Tue, 20 Jun 2023 06:20:38 GMT

Hi @Patrick Grover

We haven't heard from you since the last response from @Kaniz Fatma , and I was checking back to see if her suggestions helped you.

Or else, If you have any solution, please share it with the community, as it can be helpful to others.

Also, Please don't forget to click on the "Select As Best" button whenever the information provided helps resolve your question.

topic Connecting to Synapse database using AzureCliCredential token in Spark in Machine Learning

Connecting to Synapse database using AzureCliCredential token in Spark

Re: Connecting to Synapse database using AzureCliCredential token in Spark