12-16-2021 08:02 AM
I started to explore Databricks SQL Connector. And I'm wondering can I execute query from Databricks SQL Connector from notebook or job?
sample code:
'test_query' lives in Databricks SQL. It is ready to use query!
from databricks import sql
hostname = '<name>.databricks.com'
http_path = '/sql/1.0/endpoints/<endpoint_id>'
access_token = '<personal_token>'
connection = sql.connect(server_hostname=hostname, http_path=http_path, access_token=access_token)
cursor = connection.cursor()
cursor.execute('test_query')
result = cursor.fetchall()
for row in result:
print(row)
cursor.close()
12-16-2021 08:15 AM
except nice UI and cool features there 😉 for me biggest advantage is that from PowerBI and Qlikview I can query data using SQL and use serverless SQL endpoint.
12-16-2021 08:09 AM
yes you can as it is just python but in notebook you can use native SQL commands (of course there can be situation that you have some really big sql enpoint (like 64 DBU) so it can make sense in some situations but I don't think much and it creates complicated architecture)
12-16-2021 08:13 AM
Than that make more sense to use that in job? And what is the big benefit of Databricks SQL at all? I mean what is the main usage of Databricks SQL?
12-16-2021 08:15 AM
except nice UI and cool features there 😉 for me biggest advantage is that from PowerBI and Qlikview I can query data using SQL and use serverless SQL endpoint.
12-16-2021 08:19 AM
That means there is no advantages if I use Databricks SQL in my job
12-16-2021 08:24 AM
I would use just native SQL syntax in job but maybe there is some nice trick...
12-16-2021 08:25 AM
Thank you!
12-16-2021 11:45 AM
@Borislav Blagoev - Would you be happy to mark which of @Hubert Dudek's answers offered the best resolution as best?
12-16-2021 01:26 PM
I can't understand your question
12-17-2021 08:14 AM
@Hubert Dudek
Error during OpenSession; Request: TOpenSessionReq(client_protocol=5, username=None, password=None, configuration=None) Error: None Bounded-Retry-Delay: None Attempt: 1/30 Elapsed-Seconds: 1.3954582214355469/900.0
OperationalError: ('', EOFError())
I get the following error when I try to execute the code above. Do you know something about this?
12-17-2021 08:18 AM
@Borislav Blagoev - There is a "Select as Best" link underneath each reply. All you need to do, if you choose, is to click on that link for whichever reply best answered your question. The purpose is to help other members find helpful solutions more easily.
I'm attaching a screenshot to help.
12-17-2021 09:01 AM
Done!
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group