Are serverless endpoints possible in this Technical Blog post by qianyu?

[brandt6264]

Hi, are serverless endpoints possible for Whisper and Llama in this Technical Blog post by qianyu?

https://community.databricks.com/t5/technical-blog/streamline-customer-call-center-transcripts-analy...

Thanks!

[-werners-]

Model serving is already serverless enabled.  You can even set a budget for this (public preview).
https://docs.databricks.com/aws/en/machine-learning/model-serving/manage-serving-endpoints
https://docs.databricks.com/aws/en/machine-learning/model-serving/foundation-model-overview

Or do you mean something else (like using a databricks serverless sql/eng instance)?

[brandt6264]

Thanks for the clarification and links.

Bottom line, what I am trying to avoid is spinning up AWS resources in the background that will incur ongoing charges until I track them down and terminate them (I am a new Databricks customer and still trying to navigate the billing/cost system). The "Serve this model" UI looked suspiciously like it was going to do this but on second look, maybe not.

I am just wanting to confirm my only costs, Databricks or AWS, for Whisper and Llama in this Technical Blog post will be only for the short duration they will be used. Thanks!

[-werners-]

https://www.databricks.com/product/pricing/foundation-model-serving

We don´t serve models on databricks, but as far as i can see you pay per input/output tokens (for foundation models).
For classic models:
https://www.databricks.com/product/pricing/model-serving