topic An error occurred while loading the model. Failed to load the pickled function from a hexadecimal in Machine Learning

An error occurred while loading the model. Failed to load the pickled function from a hexadecimal

marcelo2108 — Fri, 16 Feb 2024 04:07:55 GMT

[8586fsbgpb] An error occurred while loading the model. Failed to load the pickled function from a hexadecimal string. Error: Can't get attribute 'transform_input' on <module '__main__' from '/opt/conda/envs/mlflow-env/bin/gunicorn'>.

I´m using the function to transform input and output on this way

def transform_input(**request):

print('Type of prompt',type(request["prompt"]))

request["messages"] = [

{"role": "system", "content": "You are a helpful assistant."},

{"role": "user", "content": request["prompt"]},

]

request["stop"] = ['\n\n']

print("Request format",request)

return request

def transform_output(response):

return response['candidates'][0]

# If using serving endpoint, the model serving endpoint is created in `02_[chat]_mlflow_logging_inference`

llm = Databricks(endpoint_name='llama2-7b-chat-completion',

transform_input_fn=transform_input,

transform_output_fn=transform_output,extra_params={"temperature":0.01,"max_tokens": 300})

Is there anything else I´m missing to avoid this error ?

Re: An error occurred while loading the model. Failed to load the pickled function from a hexadecima

marcelo2108 — Fri, 16 Feb 2024 12:22:32 GMT

Hi @Retired_mod I put already on top level of the cell script, exactly you mentioned as in the attachment file but no look. Should I put on the top of notebook ? Any other clue about ?

Re: An error occurred while loading the model. Failed to load the pickled function from a hexadecima

marcelo2108 — Fri, 16 Feb 2024 20:15:15 GMT

The solution I found was to create those functions in a separated python code called eg. custom_functions.py and deploy as follows in ml flow

with mlflow.start_run() as run:

signature = infer_signature(question, answer)

logged_model = mlflow.langchain.log_model(

chain,

artifact_path="chain",

registered_model_name=registered_model_name,

loader_fn=get_retriever,

persist_dir=persist_directory,

pip_requirements=["mlflow==" + mlflow.__version__,"langchain==" + langchain.__version__,"sentence_transformers","chromadb"],

code_paths=["custom_functions.py"],

#conda_env=conda_env,

input_example=question,

metadata={"task": "llm/v1/chat"},

signature=signature,

await_registration_for=900 # wait for 15 minutes for model registration to complete

)

Re: An error occurred while loading the model. Failed to load the pickled function from a hexadecima

marcelo2108 — Fri, 16 Feb 2024 20:33:50 GMT

However I could not progress in the end I mean because I found the error I reported in other thread as follows

[5bb99fzs2f] An error occurred while loading the model. You haven't configured the CLI yet! Please configure by entering `/opt/conda/envs/mlflow-env/bin/gunicorn configure`.
As described in :

Re: Problem when serving a langchain model on Data... - Databricks - 59506