topic Re: model_serving_endpoints in DAB updates every time in Data Engineering

model_serving_endpoints in DAB updates every time

Binnisb — Wed, 08 Jan 2025 01:54:46 GMT

Love the model_serving_endpoints in the dab, but now it takes over 6 minutes to deploy resources when they already exist. It says (updating) in the serving tab in the side bar even if nothing has changed.

Is there a way to not update the endpoints after initial deployment when the yml for the resource hasn't changed?

Re: model_serving_endpoints in DAB updates every time

Alberto_Umana — Wed, 08 Jan 2025 12:32:34 GMT

Hi @Binnisb,

Thanks for your question, can you provide your yml deployment file?

Re: model_serving_endpoints in DAB updates every time

VZLA — Wed, 08 Jan 2025 13:21:05 GMT

@Binnisb thanks for your question!
As far as I'm aware of, there is currently no built-in way to prevent the serving endpoints from updating during deployment, even if the YAML hasn't changed. Each deployment checks for updates and goes through the update process regardless; I believe the update step is not solely based on whether the yml for the resource has changed or not. You can reaching out to Databricks support for feature enhancement requests.

Re: model_serving_endpoints in DAB updates every time

binnisbj — Wed, 08 Jan 2025 14:34:33 GMT

resources:

model_serving_endpoints:

embed_serving:

name: embedding-gpu-m

config:

served_entities:

- name: embedding-gpu-m

entity_name: system.ai.bge_large_en_v1_5

entity_version: "3"

workload_size: Small

scale_to_zero_enabled: true

workload_type: GPU_MEDIUM

min_provisioned_throughput: 0

Re: model_serving_endpoints in DAB updates every time

Alberto_Umana — Wed, 08 Jan 2025 15:29:07 GMT

I have created an internal feature request for this behavior: DB-I-13108