cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Azure-core or AzureML version packages incompatibility

Cristhian_Plaza
New Contributor III

I'm running the BigBook of DS from Databricks on an Azure Databricks environment and I'm having a problem with a package in the first notebook inside the Integrating Azure Databricks and Azure Machine Learning folder.

To be exact, this is the problem:

ImportError: cannot import name 'DeserializationError' from 'azure.core.exceptions' (/databricks/python/lib/python3.7/site-packages/azure/core/exceptions.py)

and the notebook I have doesn't shows the specifications of how solving this problem.

The cell with the problem is:

import os
import numpy as np
import pandas as pd
import pickle
import sklearn
import joblib
import math
 
from sklearn.model_selection import train_test_split
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn_pandas import DataFrameMapper
from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score
 
import matplotlib
import matplotlib.pyplot as plt
 
import azureml
 
from azureml.core import Workspace, Experiment, Run
from azureml.core.model import Model
 
print('The azureml.core version is {}'.format(azureml.core.VERSION))

The packages installed are:

absl-py==0.9.0
adal==1.2.7
ansiwrap==0.8.4
anyio==3.6.2
applicationinsights==0.11.10
argcomplete==1.12.3
argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
asn1crypto==1.3.0
astor==0.8.0
astunparse==1.6.3
attrs==22.1.0
azure-cli-core==2.43.0
azure-cli-telemetry==1.0.8
azure-common==1.1.28
azure-core==1.26.1
azure-graphrbac==0.61.1
azure-identity==1.7.0
azure-mgmt-authorization==3.0.0
azure-mgmt-compute==29.0.0
azure-mgmt-containerregistry==10.0.0
azure-mgmt-core==1.3.2
azure-mgmt-keyvault==10.1.0
azure-mgmt-resource==21.2.1
azure-mgmt-storage==20.1.0
azure-storage-blob==12.13.0
azure-storage-queue==12.5.0
azureml==0.2.7
azureml-automl-core==1.48.0
azureml-automl-runtime==1.48.0.post1
azureml-contrib-notebook==1.48.0
azureml-core==1.48.0
azureml-dataprep==4.8.3
azureml-dataprep-native==38.0.0
azureml-dataprep-rslex==2.15.1
azureml-dataset-runtime==1.48.0
azureml-explain-model==1.48.0
azureml-interpret==1.48.0
azureml-mlflow==1.48.0
azureml-pipeline==1.48.0
azureml-pipeline-core==1.48.0
azureml-pipeline-steps==1.48.0
azureml-sdk==1.48.0
azureml-telemetry==1.48.0
azureml-train-automl==1.48.0
azureml-train-automl-client==1.48.0
azureml-train-automl-runtime==1.48.0
azureml-train-core==1.48.0
azureml-train-restclients-hyperdrive==1.48.0
azureml-training-tabular==1.48.0
azureml-widgets==1.48.0
backcall==0.1.0
backports.tempfile==1.0
backports.weakref==1.0.post1
bcrypt==3.2.0
bleach==5.0.1
blinker==1.4
bokeh==2.4.3
boto==2.49.0
boto3==1.12.0
botocore==1.15.0
bpyutils==0.5.8
cachetools==4.1.1
certifi==2020.6.20
cffi==1.14.0
chardet==3.0.4
Click==7.0
cloudpickle==1.3.0
configparser==3.7.4
contextlib2==21.6.0
cryptography==2.8
cycler==0.10.0
Cython==0.29.15
dask==2021.6.0
databricks-cli==0.11.0
dataclasses==0.6
decorator==4.4.1
defusedxml==0.7.1
dill==0.3.1.1
diskcache==5.0.2
distributed==2021.6.0
distro==1.8.0
docker==4.3.1
docutils==0.15.2
dotnetcore2==3.1.23
entrypoints==0.3
fastjsonschema==2.16.2
fire==0.5.0
Flask==1.1.1
flatbuffers==22.12.6
fsspec==2022.11.0
fusepy==3.0.1
future==0.18.2
gast==0.3.3
gensim==3.8.3
gitdb==4.0.5
GitPython==3.1.0
google-auth==1.11.2
google-auth-oauthlib==0.4.1
google-pasta==0.2.0
grpcio==1.27.2
gunicorn==20.0.4
h5py==2.10.0
HeapDict==1.0.1
horovod==0.19.5
humanfriendly==10.0
idna==2.8
importlib-metadata==4.13.0
importlib-resources==5.10.1
interpret-community==0.27.0
interpret-core==0.2.7
ipykernel==5.1.4
ipython==7.12.0
ipython-genutils==0.2.0
ipywidgets==7.7.2
isodate==0.6.0
itsdangerous==1.1.0
jedi==0.14.1
jeepney==0.8.0
Jinja2==2.11.1
jmespath==0.10.0
joblib==0.14.1
joblibspark==0.2.0
jsonpickle==2.2.0
jsonschema==4.17.3
jupyter-client==5.3.4
jupyter-core==4.6.1
jupyter-server==1.23.3
jupyterlab-widgets==1.1.1
Keras-Preprocessing==1.1.2
keras2onnx==1.6.0
kiwisolver==1.1.0
knack==0.10.1
koalas==1.2.0
lightgbm==2.3.0
llvmlite==0.38.1
locket==1.0.0
Mako==1.1.2
Markdown==3.1.1
MarkupSafe==1.1.1
matplotlib==3.1.3
mistune==0.8.4
mkl-fft==1.0.15
mkl-random==1.1.0
mkl-service==2.3.0
ml-wrappers==0.2.2
mleap==0.16.1
mlflow==1.11.0
mlflow-databricks-artifacts==2.0.1
mlflow-skinny==1.30.0
msal==1.20.0
msal-extensions==1.0.0
msgpack==1.0.4
msrest==0.7.1
msrestazure==0.6.4
nbclassic==0.4.8
nbconvert==5.6.1
nbformat==5.7.0
ndg-httpsclient==0.5.1
nest-asyncio==1.5.6
networkx==2.4
nimbusml==1.8.0
nltk==3.4.5
notebook==6.5.2
notebook-shim==0.2.2
numba==0.55.2
numpy==1.18.1
oauthlib==3.1.0
olefile==0.46
onnx==1.12.0
onnxconverter-common==1.6.0
onnxmltools==1.4.1
onnxruntime==1.11.1
opt-einsum==3.3.0
packaging==21.3
pandas==1.1.5
pandocfilters==1.5.0
papermill==1.2.1
paramiko==2.7.1
parso==0.5.2
partd==1.3.0
pathspec==0.10.3
patsy==0.5.1
petastorm==0.9.5
pexpect==4.8.0
pickleshare==0.7.5
pipupgrade==1.12.0
pkginfo==1.9.2
pkgutil-resolve-name==1.3.10
plotly==4.9.0
pmdarima==1.7.1
portalocker==2.6.0
prometheus-client==0.15.0
prompt-toolkit==3.0.3
protobuf==3.20.1
psutil==5.9.4
psycopg2==2.8.6
ptyprocess==0.6.0
pyarrow==1.0.1
pyasn1==0.4.8
pyasn1-modules==0.2.7
pycparser==2.19
Pygments==2.5.2
PyJWT==2.6.0
PyNaCl==1.3.0
pyodbc===4.0.0-unsupported
pyOpenSSL==19.1.0
pyparsing==2.4.6
pyrsistent==0.19.2
PySocks==1.7.1
python-dateutil==2.8.1
python-editor==1.0.4
pytz==2019.3
PyYAML==5.3.1
pyzmq==18.1.1
querystring-parser==1.2.4
requests==2.22.0
requests-oauthlib==1.3.0
retrying==1.3.3
rsa==4.0
s3transfer==0.3.3
scikit-learn==0.22.2.post1
scipy==1.5.3
seaborn==0.10.0
SecretStorage==3.3.3
Send2Trash==1.8.0
shap==0.39.0
simplejson==3.17.0
six==1.14.0
skl2onnx==1.4.9
sklearn-pandas==1.7.0
slicer==0.0.7
smart-open==1.9.0
smmap==3.0.4
sniffio==1.3.0
sortedcontainers==2.4.0
spark-tensorflow-distributor==0.1.0
sqlparse==0.4.3
statsmodels==0.11.0
tabulate==0.8.3
tblib==1.7.0
tenacity==8.1.0
tensorboard==2.3.0
tensorboard-plugin-wit==1.7.0
tensorflow-cpu==2.3.0
tensorflow-estimator==2.3.0
termcolor==1.1.0
terminado==0.17.1
testpath==0.6.0
textwrap3==0.9.2
threadpoolctl==3.1.0
toolz==0.12.0
torch==1.6.0
torchvision==0.7.0
tornado==6.0.3
tqdm==4.42.1
traitlets==4.3.3
typing-extensions==4.4.0
urllib3==1.25.8
wcwidth==0.1.8
webencodings==0.5.1
websocket-client==0.56.0
Werkzeug==1.0.0
widgetsnbextension==3.6.1
wrapt==1.11.2
xgboost==1.1.1
zict==2.2.0
zipp==3.11.0

These packages are the ones installed by default on a ML single node cluster

5 REPLIES 5

Kaniz_Fatma
Community Manager
Community Manager

Hi @Cristhian Plazas​, Here is a link for troubleshooting the DeserializationError. Please let us know if that helps.

Hi Kaniz, this approach didn't work and it's obvious. In the docs that you kindly provided me, they make an update of the azureml-sdk package for one highest than 1.42 but if you see my list of packages the version of mine is 1.48. Even so, I ran the installation they mentioned but it didn't work.

Thank you @Cristhian Plazas​ for your response. We'll bring another solution for you.

Kaniz_Fatma
Community Manager
Community Manager

Hi @Cristhian Plazas​, Please upgrade your azure-core package and let us know if that works.

Here is a GitHub link where this issue has been addressed. Please have a look at the conversations.

Hi @Kaniz Fatma​, it failed. Just to be clear the VM is a Standard_DS3_v2 7.3 LTS ML with Apache Spark 3.0.1 and Scala 2.12. I'm trying just to reproduce an excercise of the BigBook of ML by Databricks. 

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!