cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Issue connecting to SQL warehouse spark thrift server

AlexDavies
Contributor

we have a library that allows dotnet applications to talk to databricks clusters (https://github.com/clearbank/SparkSqlClient). This communicates with the clusters over the spark thrift server

Although this works great for clusters in the "data science and engineering" workspace it cant seem to connect to any of the sql warehouse clusters

Do the sql warehouse clusters use a different spark sql thrift server than the other clusters?

Where can I get a copy of the thrift file that describes the endpoint that sql warehouses endpoints support?

2 REPLIES 2

Anonymous
Not applicable

@Alex Davies​ It should be the same thrift server for SQL warehouse.

You can find the connection URLs in connection details tab of the SQL warehouse. Also Databricks provides native connector for multiple partner tools. Please refer here

image

AlexDavies
Contributor

I have tried those connection details however it they give me 400 errors when trying to connect directly using the hive thrift server contract (https://github.com/apache/hive/blob/master/service-rpc/if/TCLIService.thrift). I do not get the issues when connecting to data science and engineering clusters. Do the sql warehouse use a slightly different contract? Could I get access to the thrift file the sql warehouses use?

I know there are lots of connectors, however there are no native ones for dotnet. There is an ODBC driver but requires a level of environment setup that is not always easy possible on cloud infrastructure. Which is why I was writing a a native dotnet connector, however stuck on determining what the contract is for SQL warehouses and how it differs from normal clusters

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.