Serving pay-per-token Chat LLM Model

Henrik — Wed, 28 Aug 2024 06:17:18 GMT

We have build a chat solution on LLM RAG chat model, but we face an issue when we spin up a service endpoint to host the model.

According to the documentation, there should be sevral LLM models available as pay-per-token endpoints, for instance the DBRX Instruct.

https://learn.microsoft.com/en-us/azure/databricks/machine-learning/foundation-models/supported-models

However, in our workspace we only se two available pay-per-token endpoints (se attachment "serving endpoints.png").

When we "create a new service endpoint", it seems like we can only spin up "provisioned throughtput models, which are currently too expensive to run for our setup (se attachment "issue.png").

Our Databricks environment is in azure west europe.

Any suggestions?

Re: Serving pay-per-token Chat LLM Model

daniel_sahal — Thu, 29 Aug 2024 05:26:43 GMT

@Henrik
The documentation clearly states that it should be available in west europe, but i'm also unable to see DBRX ppt endpoint.
I think that it would be best to raise an Azure Support ticket - they should either somehow enable it on your workspace or modify the documentation.

topic Re: Serving pay-per-token Chat LLM Model in Get Started Discussions

Serving pay-per-token Chat LLM Model

Re: Serving pay-per-token Chat LLM Model