Your assumption that minimizing inference lag is more important than minimizing the size of the model in a distributed setting is generally correct.

In a distributed environment, models are typically loaded once per worker, as you mentioned, which means that the impact of model size is limited to the initial loading of the model. However, inference lag occurs every time an observation is processed, which can have a significant impact on the overall performance of the system.