Databricks Community

HemantvIkani32 · 4 weeks ago

I am facing this below error while serving codellama model:
Exception: Request failed with status 400, {"error_code":"INVALID_PARAMETER_VALUE","message":"The provisioned throughput model Llama 2 7B is deprecated and no longer supported in serving. See https://docs.databricks.com/aws/en/machine-learning/foundation-model-apis/supported-models for information on currently supported models.","details":[{"@type":"type.googleapis.com/google.rpc.RequestInfo","request_id":"ff0707d9-1c8d-4b82-b34f-c6d138709b70","serving_data":""}]}

I just wanted to know the method by which databricks identifies the model whether it is deprecated or not. Does this check the model name or config?

szymon_dybczak · 4 weeks ago

Hi @HemantvIkani32 ,

Databricks is always publicly announcing deprecation of any model. It's well described here:

Generative AI models maintenance policy | Databricks on AWS

And here you can find list of retired models:

Generative AI models maintenance policy | Databricks on AWS

HemantvIkani32 · 4 weeks ago

Hi @szymon_dybczak ,

Thanks for the response, Suppose I register a model (say Llama2‑7B) in Databricks using some custom name. How does Databricks “know” that this model is actually Llama2‑7B under the hood? Is the identification based on configuration files, tokenizer metadata, or something else?

szymon_dybczak · 4 weeks ago

I guess they identify the model based on some kind of metadata. This is an implementation detail though, so unless someone from the Databricks team joins the discussion, we can only guess.

jamesl · 3 weeks ago

Hi @HemantvIkani32 , the models are packaged as MLflow models which contain all the metadata required to identify the underlying model and serving/env requirements. If you didn't see already, there are also see details about the CodeLlama models in the Databricks Marketplace: https://marketplace.databricks.com/details/4b0b9034-0f9b-4d25-ac7c-52ca956af838/Databricks_CodeLlama...