cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

upstream request timeout in databricks apps when using databricks sql connector

yathish
New Contributor II

Hi, i am building an application in Databricks apps, Sometimes when i try to fetch data using Databricks SQL connector in an API, it takes time to hit the SQL warehouse and if the time exceeds more than 60 seconds it gives upstream timeout error. I have monitored the queries once they reach the warehouse it gets executed in milliseconds. The time being taken is to send the query to warehouse for execution and to send back the data to the server. how can this be handled? also this happens only some times.

5 REPLIES 5

Alberto_Umana
Databricks Employee
Databricks Employee

Hi @yathish,

When you submit the query to the warehouse, is the warehouse up and running already? and are you using classic, pro or serverless warehouse?

yathish
New Contributor II

Hi @Alberto_Umana, yes the server is up and running. I am using serverless warehouse. Thank you!

 

yathish
New Contributor II

def credential_provider():
config = Config(
host=os.getenv('DATABRICKS_HOST'),
client_id=os.getenv("DATABRICKS_CLIENT_ID"),
client_secret=os.getenv("DATABRICKS_CLIENT_SECRET"))
return oauth_service_principal(config)


def db_connector():
connection = sql.connect(
server_hostname=os.getenv('DATABRICKS_HOST'),
http_path=os.getenv('WAREHOUSE_HTTP_PATH'),
credentials_provider=credential_provider)

return connection



and in the api i am using it as

with db_connector() as connection:
     with connection.cursor() as cursor:
          results = cursor.execute(query).fetchall()

renatagotler
New Contributor II

It is happening to us too! We have a model that call the sql connection and we are getting a lot of this error in the serving endpoint. We also use serveless.

 

renatagotler_0-1740054303062.png

 

yathish
New Contributor II

@Alberto_Umana Can you please let us know if there are any updates on this?

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now