Goal:
Let's say we have a custom function with arbitrary python code . And we want to deploy that as model as we wanted to generate an API endpoint .So that that can be accessed from external applications.
I am attaching the below code I have tried .If you know any better way to do this .Please suggest.
Sample Code I tried:
import mlflow
import mlflow.pyfunc
import pandas as pd
from mlflow.models.signature import infer_signature
class Model(mlflow.pyfunc.PythonModel):
def cust_func():
# Perform the SQL operation on the driver
sql_query_insert = f"""INSERT INTO {FULL_TABLE_NAME_FOR_LOAD} (type) SELECT 'Sandip'"""
spark.sql(sql_query_insert)
def predict(self, context, model_input):
cust_func()
return pd.DataFrame(["Done"], columns=["result"])
# Convert the input data to a pandas DataFrame
question = pd.DataFrame([{"question": "Please do your task"}])
with mlflow.start_run(run_name="A_run_created_from_notebook2") as run:
model = Model()
signature = infer_signature(question, model.predict(None, question))
mlflow.pyfunc.log_model("A_model_created_from_notebook2", python_model=model, signature=signature)
Error:
AT API endpoint if we pass request :
{"inputs" : "Please do the task"}
Nothing is coming out.
Error in console : 2024/08/25 05:25:08 WARNING mlflow.pyfunc.scoring_server: If using `instances` as input key, we internally convert the data type from `records` (List[Dict]) type to `list` (Dict[str, List]) type if the data is a pandas dataframe representation. This might cause schema changes. Please use `inputs` to avoid this converesion.
Sandip Bhowmick
