topic Re: LLAMA3.1 logging using mlflow in Generative AI

LLAMA3.1 logging using mlflow

HemantvIkani32 — Wed, 03 Sep 2025 14:59:09 GMT

Hi team, I am facing issue after registering the meta llama 3.1 8b instruct model, while serving i get the response like

{error code: BAD REQUEST, "message": "Time out while evaluating the model. Verify that the model evaluate within the timeframe."

Given below my code snippet (IF there is any other way to register please help):

input_schema = Schema([ColSpec("string", "prompt"),

ColSpec("double", "temperature", required=False),

ColSpec("integer", "max_tokens", required=False),

ColSpec("string", "stop", required=False),

ColSpec("integer", "candidate_count", required=False)])

output_schema = Schema([ColSpec('string', 'predictions')])

signature = ModelSignature(inputs=input_schema, outputs=output_schema)

input_example = {"prompt": "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\nWhat is Apache Spark?\n\n### Response:\n"}

mlflow.set_registry_uri("databricks")

with mlflow.start_run() as run:

result = mlflow.transformers.log_model(transformers_model={ "model": model,"tokenizer": tokenizer},

task = "text-generation",

artifact_path=REGISTERED_MODEL_NAME,

registered_model_name=REGISTERED_MODEL_NAME,

signature=signature,

input_example=input_example,

await_registration_for = 3600)

Re: LLAMA3.1 logging using mlflow

ilir_nuredini — Thu, 04 Sep 2025 08:20:20 GMT

Hi @HemantvIkani32 ,

Comparing your result and looking into this documentation: https://learn.microsoft.com/en-us/azure/databricks/machine-learning/model-serving/model-serving-timeouts,
the error message indicates that requests took longer than the default timeouts for MLflow environment variables (one of the reasons).
Please follow some of the suggested approaches in the given doc and see if that fixes the issue.

Hope that helps,

Best, Ilir

Re: LLAMA3.1 logging using mlflow

Advika — Thu, 25 Sep 2025 14:41:15 GMT

Hello @HemantvIkani32!

Did the above suggestion help resolve your issue? If yes, please consider marking it as the accepted solution.
If you found another approach, it would be great if you could share it with the community.