cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Issue connecting to SQL warehouse spark thrift server

AlexDavies
Contributor

we have a library that allows dotnet applications to talk to databricks clusters (https://github.com/clearbank/SparkSqlClient). This communicates with the clusters over the spark thrift server

Although this works great for clusters in the "data science and engineering" workspace it cant seem to connect to any of the sql warehouse clusters

Do the sql warehouse clusters use a different spark sql thrift server than the other clusters?

Where can I get a copy of the thrift file that describes the endpoint that sql warehouses endpoints support?

2 REPLIES 2

Anonymous
Not applicable

@Alex Davies​ It should be the same thrift server for SQL warehouse.

You can find the connection URLs in connection details tab of the SQL warehouse. Also Databricks provides native connector for multiple partner tools. Please refer here

image

AlexDavies
Contributor

I have tried those connection details however it they give me 400 errors when trying to connect directly using the hive thrift server contract (https://github.com/apache/hive/blob/master/service-rpc/if/TCLIService.thrift). I do not get the issues when connecting to data science and engineering clusters. Do the sql warehouse use a slightly different contract? Could I get access to the thrift file the sql warehouses use?

I know there are lots of connectors, however there are no native ones for dotnet. There is an ODBC driver but requires a level of environment setup that is not always easy possible on cloud infrastructure. Which is why I was writing a a native dotnet connector, however stuck on determining what the contract is for SQL warehouses and how it differs from normal clusters

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group