cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Issue connecting to SQL warehouse spark thrift server

AlexDavies
Contributor

we have a library that allows dotnet applications to talk to databricks clusters (https://github.com/clearbank/SparkSqlClient). This communicates with the clusters over the spark thrift server

Although this works great for clusters in the "data science and engineering" workspace it cant seem to connect to any of the sql warehouse clusters

Do the sql warehouse clusters use a different spark sql thrift server than the other clusters?

Where can I get a copy of the thrift file that describes the endpoint that sql warehouses endpoints support?

2 REPLIES 2

Anonymous
Not applicable

@Alex Daviesโ€‹ It should be the same thrift server for SQL warehouse.

You can find the connection URLs in connection details tab of the SQL warehouse. Also Databricks provides native connector for multiple partner tools. Please refer here

image

AlexDavies
Contributor

I have tried those connection details however it they give me 400 errors when trying to connect directly using the hive thrift server contract (https://github.com/apache/hive/blob/master/service-rpc/if/TCLIService.thrift). I do not get the issues when connecting to data science and engineering clusters. Do the sql warehouse use a slightly different contract? Could I get access to the thrift file the sql warehouses use?

I know there are lots of connectors, however there are no native ones for dotnet. There is an ODBC driver but requires a level of environment setup that is not always easy possible on cloud infrastructure. Which is why I was writing a a native dotnet connector, however stuck on determining what the contract is for SQL warehouses and how it differs from normal clusters

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group