topic Re: SQL Warehouse connections in us-east-2 in Data Engineering

SQL Warehouse connections in us-east-2

marcus1 — Fri, 30 Sep 2022 19:13:26 GMT

I have 2 very similarly configured workspaces, one in us-west-2 and one in us-east-2.

Both got configured by default with a "Starter Warehouse".

The one in us-west-2 I can reach via the internet using python databricks-sql-connector, but the one in us-east-2 I can not. I performed a test on a workspace in us-east-1 and was able to connect.

Interestingly however the workspace in us-east-2 can query the datasources using the same "Starter Warehouse", with appropriate results.

TIA

Re: SQL Warehouse connections in us-east-2

Hubert-Dudek — Fri, 30 Sep 2022 19:17:23 GMT

Maybe routing is configured differently in your VNET. That SQL warehouse, after all, is a machine in your VNET.Yyou can preview everything in resources in the azure portal (or AWS). As it is two different regions, it will be diffrent vents, and other rules can apply (routing table, etc.)

Re: SQL Warehouse connections in us-east-2

marcus1 — Fri, 30 Sep 2022 20:41:01 GMT

The workspaces were created identically with the CloudFormation automation provided by databricks. So both workspaces for instance allow access via DB cli and api, so the issue of routing from the public internet is not an issue, at least to the workspace.

When I curl the sql endpoint both respond.

But with the us-east-2 server:

Traceback (most recent call last):

File "/Users/marcussimonsen/work/analytics/db_sql_connector.py", line 4, in <module>

connection = sql.connect(

File "/Users/marcussimonsen/work/analytics/.venv/lib/python3.8/site-packages/databricks/sql/__init__.py", line 50, in connect

return Connection(server_hostname, http_path, access_token, **kwargs)

File "/Users/marcussimonsen/work/analytics/.venv/lib/python3.8/site-packages/databricks/sql/client.py", line 129, in __init__

self._session_handle = self.thrift_backend.open_session(

File "/Users/marcussimonsen/work/analytics/.venv/lib/python3.8/site-packages/databricks/sql/thrift_backend.py", line 444, in open_session

response = self.make_request(self._client.OpenSession, open_session_req)

File "/Users/marcussimonsen/work/analytics/.venv/lib/python3.8/site-packages/databricks/sql/thrift_backend.py", line 373, in make_request

self._handle_request_error(error_info, attempt, elapsed)

File "/Users/marcussimonsen/work/analytics/.venv/lib/python3.8/site-packages/databricks/sql/thrift_backend.py", line 241, in _handle_request_error

raise network_request_error

databricks.sql.exc.RequestError: Error during request to server

Re: SQL Warehouse connections in us-east-2

Sivaprasad1 — Mon, 03 Oct 2022 14:01:29 GMT

@Marcus Simonsen

could you check connectivity to endpoint from where you are running this code?

from linux terminal

nc -zv <Workspacename>.cloud.databricks.com 443 (Replace workspacename)

Re: SQL Warehouse connections in us-east-2

marcus1 — Tue, 04 Oct 2022 15:09:36 GMT

Connection suceeds. Curl to the SQL endpoint is also fine.

Re: SQL Warehouse connections in us-east-2

marcus1 — Tue, 04 Oct 2022 15:11:14 GMT

I noticed somthing in https://status.databricks.com/ that SQLServerless isn't operational in us-east-2. Can anyone confirm this?

Re: SQL Warehouse connections in us-east-2

Hubert-Dudek — Wed, 12 Oct 2022 13:16:43 GMT

Now they are https://docs.databricks.com/release-notes/product/2022/october.html#serverless-sql-warehouses-are-available-in-regions-eu-central-1-and-us-east-2.

So it seems that was a problem because they were not yet available when you created the post, and it was some transition period.

Re: SQL Warehouse connections in us-east-2

marcus1 — Fri, 14 Oct 2022 17:21:05 GMT

Thanks Hubert! Still having the same exception as above. As a last resort I'll create another workspace in us-east-2 to see if I can reproduce.

Re: SQL Warehouse connections in us-east-2

Anonymous — Sun, 23 Oct 2022 06:04:54 GMT

Hi @Marcus Simonsen

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help.

We'd love to hear from you.

Thanks!

Re: SQL Warehouse connections in us-east-2

marcus1 — Wed, 09 Nov 2022 16:27:04 GMT

Yes, newer clusters/warehouses spun up, are now reachable in us-east-2.