TypeError: ColSpec.init() got an unexpected keyword argument 'required'

Sangeethagk — Mon, 17 Jun 2024 10:56:39 GMT

Hi Team, one of my customer is facing the below issue.. Anyone faced this issue before ? Any help would be appreciated.

import mlflow

mlflow.set_registry_uri("databricks-uc")

catalog_name = "system"

embed = mlflow.pyfunc.spark_udf(spark, f"models:/system.ai.bge_m3/1", "array<float>")

On running the above piece of code, we are getting the below error

TypeError: ColSpec.__init__() got an unexpected keyword argument 'required'

WARNING mlflow.pyfunc: Detected one or more mismatches between the model's dependencies and the current Python environment: - mlflow (current: 2.7.1, required: mlflow==2.11.2) - torch (current: 2.0.1+cu118, required: torch==2.2.1) - transformers (current: 4.31.0, required: transformers==4.38.2) To fix the mismatches, call `mlflow.pyfunc.get_model_dependencies(model_uri)` to fetch the model's environment and install dependencies using the resulting environment file.

WARNING mlflow.pyfunc: Calling `spark_udf()` with `env_manager="local"` does not recreate the same environment that was used during training, which may lead to errors or inaccurate predictions. We recommend specifying `env_manager="conda"`, which automatically recreates the environment that was used to train the model and performs inference in the recreated environment.

Re: TypeError: ColSpec.init() got an unexpected keyword argument 'required'

viksuper555 — Tue, 15 Oct 2024 15:20:44 GMT

Upgrade the version of the mlflow package. In 2.7.1 there is no such parameter.
https://mlflow.org/docs/2.7.1/python_api/mlflow.types.html
While in the latest (2.17.0) there is such https://mlflow.org/docs/2.17.0/python_api/mlflow.types.html

%pip install mlflow==2.17.0 dbutils.library.restartPython()

topic TypeError: ColSpec.__init__() got an unexpected keyword argument 'required' in Data Engineering

TypeError: ColSpec.__init__() got an unexpected keyword argument 'required'

Re: TypeError: ColSpec.__init__() got an unexpected keyword argument 'required'

topic TypeError: ColSpec.init() got an unexpected keyword argument 'required' in Data Engineering

TypeError: ColSpec.init() got an unexpected keyword argument 'required'

Re: TypeError: ColSpec.init() got an unexpected keyword argument 'required'