topic Provisioned concurrency of serving endpoints scales to zero in Machine Learning

Provisioned concurrency of serving endpoints scales to zero

chidifrank — Wed, 02 Aug 2023 09:28:17 GMT

Hi,

We provisioned the endpoint with 4 DBUs and also disabled the scale_to_zero option. For some reason, it randomly drops to 0 provisioned concurrency. Logs available in the serving endpoint service are not insightful.

Currently, we are provisioning the endpoint with 8 DBUs but still, it randomly drops 4. What might be the issue?

Re: Provisioned concurrency of serving endpoints scales to zero

chidifrank — Thu, 03 Aug 2023 11:45:32 GMT

Hi,

I apologize if my question wasn't clear; let me clarify it.
We are not using the scale_to_zero option and we are not doing any warmup requests so it should never scale to zero despite traffic or zero traffic right?