Timeout on docker pull in Databricks Container Services

[kch]

Hello,

There is a timeout that limits the size of images used in Docker Container Service. When using images containing large ML libraries, the size often exceeds the limit that could be pulled. Is there any plan to add parametrization of this timeout? Or at least increase it?

[Walter_C]

Currently, the timeout for pulling Docker images in Databricks Container Service is fixed at 10 minutes and cannot be changed.

The primary recommendation to address this issue is to reduce the size of your Docker image. This can be achieved by flattening the Docker image to minimize the number of layers, as each layer must be written to disk sequentially, which can create a bottleneck.

[kch]

@Walter_C Thank you for your response. Unfortunately, this doesn't resolve my issue. Using the GPU version of Torch even without additional dependencies comes very close to the limit. Once I add the necessary components for training or using the ML model, it’s easy to exceed the limit, even with everything computed in a single layer, caches cleared, and dependencies optimized. Considering how common this situation has become, I’m curious if there are any plans to parameterize or adjust the limit?

[kch]

Are there any new or planned changes in the policy?