topic Re: upstream request timeout in databricks apps when using databricks sql connector in Data Engineering

upstream request timeout in databricks apps when using databricks sql connector

yathish — Fri, 14 Feb 2025 17:56:02 GMT

Hi, i am building an application in Databricks apps, Sometimes when i try to fetch data using Databricks SQL connector in an API, it takes time to hit the SQL warehouse and if the time exceeds more than 60 seconds it gives upstream timeout error. I have monitored the queries once they reach the warehouse it gets executed in milliseconds. The time being taken is to send the query to warehouse for execution and to send back the data to the server. how can this be handled? also this happens only some times.

Re: upstream request timeout in databricks apps when using databricks sql connector

Alberto_Umana — Sun, 16 Feb 2025 21:47:18 GMT

Hi @yathish,

When you submit the query to the warehouse, is the warehouse up and running already? and are you using classic, pro or serverless warehouse?

Re: upstream request timeout in databricks apps when using databricks sql connector

yathish — Mon, 17 Feb 2025 18:49:03 GMT

Hi @Alberto_Umana, yes the server is up and running. I am using serverless warehouse. Thank you!

Re: upstream request timeout in databricks apps when using databricks sql connector

yathish — Mon, 17 Feb 2025 18:53:30 GMT

def credential_provider():
config = Config(
host=os.getenv('DATABRICKS_HOST'),
client_id=os.getenv("DATABRICKS_CLIENT_ID"),
client_secret=os.getenv("DATABRICKS_CLIENT_SECRET"))
return oauth_service_principal(config)

def db_connector():
connection = sql.connect(
server_hostname=os.getenv('DATABRICKS_HOST'),
http_path=os.getenv('WAREHOUSE_HTTP_PATH'),
credentials_provider=credential_provider)

return connection

and in the api i am using it as

with db_connector() as connection:
with connection.cursor() as cursor:
results = cursor.execute(query).fetchall()

Re: upstream request timeout in databricks apps when using databricks sql connector

renatagotler — Thu, 20 Feb 2025 12:39:09 GMT

It is happening to us too! We have a model that call the sql connection and we are getting a lot of this error in the serving endpoint. We also use serveless.

Re: upstream request timeout in databricks apps when using databricks sql connector

yathish — Thu, 20 Feb 2025 18:33:57 GMT

@Alberto_Umana Can you please let us know if there are any updates on this?

Re: upstream request timeout in databricks apps when using databricks sql connector

epistoteles — Tue, 10 Jun 2025 07:57:16 GMT

@Alberto_Umana Any news on this? I am having similar issues and am also using a (running) serverless SQL warehouse.