Model deprecation issue while serving on Databricks

[HemantvIkani32]

I am facing this below error while serving codellama model:
Exception: Request failed with status 400, {"error_code":"INVALID_PARAMETER_VALUE","message":"The provisioned throughput model Llama 2 7B is deprecated and no longer supported in serving. See https://docs.databricks.com/aws/en/machine-learning/foundation-model-apis/supported-models for information on currently supported models.","details":[{"@type":"type.googleapis.com/google.rpc.RequestInfo","request_id":"ff0707d9-1c8d-4b82-b34f-c6d138709b70","serving_data":""}]}

I just wanted to know the method by which databricks identifies the model whether it is deprecated or not. Does this check the model name or config?

[szymon_dybczak]

Hi @HemantvIkani32 ,

Databricks is always publicly announcing deprecation of any model. It's well described here:

Generative AI models maintenance policy | Databricks on AWS

And here you can find list of retired models:

Generative AI models maintenance policy | Databricks on AWS

[HemantvIkani32]

Hi @szymon_dybczak ,

Thanks for the response, Suppose I register a model (say Llama2‑7B) in Databricks using some custom name. How does Databricks “know” that this model is actually Llama2‑7B under the hood? Is the identification based on configuration files, tokenizer metadata, or something else?

[szymon_dybczak]

I guess they identify the model based on some kind of metadata. This is an implementation detail though, so unless someone from the Databricks team joins the discussion, we can only guess.

[jamesl]

Hi @HemantvIkani32 , the models are packaged as MLflow models which contain all the metadata required to identify the underlying model and serving/env requirements. If you didn't see already, there are also see details about the CodeLlama models in the Databricks Marketplace: https://marketplace.databricks.com/details/4b0b9034-0f9b-4d25-ac7c-52ca956af838/Databricks_CodeLlama...

[Advika]

Hello @HemantvIkani32!

Did the responses shared above help resolve your concern? If yes, please consider marking the relevant response(s) as the accepted solution.