cancel
Showing results for 
Search instead for 
Did you mean: 
Community Platform Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Showing results for 
Search instead for 
Did you mean: 

Databricks app giving 'upstream request timeout '

Rakeshch007
New Contributor

Hello all,

We are developing an app which is based on flask, which is used to download logs from databricks dbfs location. For this useful case we are using databricks inbuilt App feature to deploy our app.

While we pass a smaller file it is getting downloaded without any error, but if we pass a bigger file like 300-400 mb it is giving 'upstream request timeout' after exactly one minute. But when we ran it in local it is working fine.

So we think it may be because of proxy timeout. But we don't have access to our databricks proxy. How can we increase this timeout for the proxy within the app or using env variables.

Please help me with this

 

1 REPLY 1

Isi
New Contributor II

Hey!

It looks like the issue you’re facing might be related to the proxy timeout when downloading large files from DBFS. Since modifying the proxy settings might not be an option, there are a couple of alternative approaches you could consider to mitigate this issue.

First, instead of downloading the entire file at once, you can try downloading it in smaller chunks to avoid hitting the request timeout limit. By implementing chunked downloads, you can bypass proxy-imposed limits and improve reliability.

Another effective approach is to store logs directly in an external cloud storage solution, such as AWS S3, Azure Blob Storage, or Google Cloud Storage, instead of DBFS. Databricks allows you to configure cluster logging to automatically save logs to these cloud storage solutions, making them easier to access and download without relying on DBFS.

🙂




Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group