Sunday
Hello,
I am trying to explore triggering for sql queries from Databricks notebook to serverless sql warehouse along with nest-asyncio module.
Both the above are very new for me and need help on the same.
For triggering the API from notebook, I am using the link to avoid the authentication issues, w.queries: Queries โ Databricks SDK for Python beta documentation
When I try to execute the query, it says, AttributeError: module 'databricks.sdk.service.sql' has no attribute 'CreateQueryRequestQuery'.
Can anyone shed some light with an example if you have tried this option.
Also let me know if any better option available.
Thanks
Sathya
Monday
Hi @sathya08 ,
Are you using Databricks SDK version v0.28.0 or older? I believe the CreateQueryRequestQuery was only added in recent releases (v0.28.0+), so older versions would fail with such error. Can you please try upgrading the sdk?
Check the current version:
Monday
Yes, thanks, the error resolved after the upgrade. I have trying to execute_statement from notebook to the sql warehouse as below,
Tuesday
can anyone help here.
Thanks
Sathya
Tuesday
Hi @sathya08
I understood that you want to submit an SQL query from a notebook to a serverless cluster.
Maybe you can try the below approach to achieve this.
You need to install the below package:
%pip install databricks-sql-connector
yesterday
Hi Amit, Thanks for your reply and the steps. I have already tried this option and its working. The main reason for me to go to sdk was to avoid the authentication part.
Would you be able to help in with the understanding on the *do* part of the function which is erroring out.
Thanks again for your help
Sathya
yesterday
Hi @sathya08,
Maybe you can try the below one.
I can execute the query and get the output without authenticating to the SQL Warehouse cluster.
yesterday
Thanks Amit,
I have a dumb question to ask, since I am mentioning the token is that not part of authentication ? because you mentioned that this is without the authentication. please clarify.
Another one, how can I make sure that this query runs on sql warehouse, When I check the Queries tab in databricks, I don't see any new query executed.
Thanks so much for your help
Sathya
yesterday
Hi @sathya08,
I'm not re-authenticating to the SQL server serverless for the authentication, but a one-time authentication to the workspace is required in my case. Without credentials, I'm getting below error.
default auth: cannot configure default credentials
Looks like, if we don't provide any credential details it will search for the default credential set at my profile level.
For the query execution log, you can check in the Query History tab. Make sure you remove the filter from the user to see the complete logs.
Thanks
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโt want to miss the chance to attend and share knowledge.
If there isnโt a group near you, start one and help create a community that brings people together.
Request a New Group