cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Hi, I get the following error when I enable model serving for spacy model via MLFLOW.

harish_s
New Contributor II

+ echo 'GUNICORN_CMD_ARGS=--timeout 63 --workers 4 '

GUNICORN_CMD_ARGS=--timeout 63 --workers 4 

+ mlflow models serve --no-conda -m /tmp/tmp1a4ltdrk/spacymodelv1 -h unix:/tmp/3.sock -p1

2022/03/01 08:26:37 INFO mlflow.models.cli: Selected backend for flavor 'python_function'

2022/03/01 08:26:37 INFO mlflow.pyfunc.backend: === Running command 'gunicorn --timeout=60 -b unix:/tmp/3.sock:1 -w 1 ${GUNICORN_CMD_ARGS} -- mlflow.pyfunc.scoring_server.wsgi:app'

[2022-03-01 08:26:38 +0000] [24589] [INFO] Starting gunicorn 20.1.0

[2022-03-01 08:26:38 +0000] [24589] [INFO] Listening at: unix:/tmp/3.sock:1 (24589)

[2022-03-01 08:26:38 +0000] [24589] [INFO] Using worker: sync

[2022-03-01 08:26:38 +0000] [24591] [INFO] Booting worker with pid: 24591

[2022-03-01 08:26:38 +0000] [24592] [INFO] Booting worker with pid: 24592

[2022-03-01 08:26:38 +0000] [24593] [INFO] Booting worker with pid: 24593

[2022-03-01 08:26:38 +0000] [24594] [INFO] Booting worker with pid: 24594

[2022-03-01 08:26:40 +0000] [24593] [ERROR] Exception in worker process

Traceback (most recent call last):

 File "/databricks/conda/envs/model-3/lib/python3.8/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker

  worker.init_process()

 File "/databricks/conda/envs/model-3/lib/python3.8/site-packages/gunicorn/workers/base.py", line 134, in init_process

  self.load_wsgi()

 File "/databricks/conda/envs/model-3/lib/python3.8/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi

  self.wsgi = self.app.wsgi()

 File "/databricks/conda/envs/model-3/lib/python3.8/site-packages/gunicorn/app/base.py", line 67, in wsgi

  self.callable = self.load()

 File "/databricks/conda/envs/model-3/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 58, in load

  return self.load_wsgiapp()

 File "/databricks/conda/envs/model-3/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp

  return util.import_app(self.app_uri)

 File "/databricks/conda/envs/model-3/lib/python3.8/site-packages/gunicorn/util.py", line 359, in import_app

  mod = importlib.import_module(module)

 File "/databricks/conda/envs/model-3/lib/python3.8/importlib/__init__.py", line 127, in import_module

  return _bootstrap._gcd_import(name[level:], package, level)

 File "<frozen importlib._bootstrap>", line 1014, in _gcd_import

 File "<frozen importlib._bootstrap>", line 991, in _find_and_load

 File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked

 File "<frozen importlib._bootstrap>", line 671, in _load_unlocked

 File "<frozen importlib._bootstrap_external>", line 783, in exec_module

 File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed

 File "/databricks/conda/envs/model-3/lib/python3.8/site-packages/mlflow/pyfunc/scoring_server/wsgi.py", line 6, in <module>

  app = scoring_server.init(load_model(os.environ[scoring_server._SERVER_MODEL_PATH]))

 File "/databricks/conda/envs/model-3/lib/python3.8/site-packages/mlflow/pyfunc/__init__.py", line 710, in load_model

  model_impl = importlib.import_module(conf[MAIN])._load_pyfunc(data_path)

 File "/databricks/conda/envs/model-3/lib/python3.8/site-packages/mlflow/pyfunc/model.py", line 269, in _load_pyfunc

  python_model = cloudpickle.load(f)

 File "stringsource", line 6, in spacy.pipeline.trainable_pipe.__pyx_unpickle_TrainablePipe

_pickle.PickleError: Incompatible checksums (68673003 vs 0x61fbab5 = (cfg, model, name, scorer, vocab))

[2022-03-01 08:26:40 +0000] [24593] [INFO] Worker exiting (pid: 24593)

[2022-03-01 08:26:41 +0000] [24594] [ERROR] Exception in worker process

1 ACCEPTED SOLUTION

Accepted Solutions

Prabakar
Esteemed Contributor III

Hi @Harish S​ this error could happen if the backend services are not updated. Are you doing this test in a PVC environment or a standard workspace?

View solution in original post

3 REPLIES 3

Hi @Kaniz Fatma​ 

Thanks for the reply, the version of the Spacy installed is 3.1.3.

Please find the attached details for the same:

image 

I will be very thankful, if I get some help here.

Which DBR version are you using? also, have you check if the DBR has this library/version?

Prabakar
Esteemed Contributor III

Hi @Harish S​ this error could happen if the backend services are not updated. Are you doing this test in a PVC environment or a standard workspace?

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group