12-15-2022 12:54 PM
I'm running the BigBook of DS from Databricks on an Azure Databricks environment and I'm having a problem with a package in the first notebook inside the Integrating Azure Databricks and Azure Machine Learning folder.
To be exact, this is the problem:
ImportError: cannot import name 'DeserializationError' from 'azure.core.exceptions' (/databricks/python/lib/python3.7/site-packages/azure/core/exceptions.py)
and the notebook I have doesn't shows the specifications of how solving this problem.
The cell with the problem is:
import os
import numpy as np
import pandas as pd
import pickle
import sklearn
import joblib
import math
from sklearn.model_selection import train_test_split
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn_pandas import DataFrameMapper
from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score
import matplotlib
import matplotlib.pyplot as plt
import azureml
from azureml.core import Workspace, Experiment, Run
from azureml.core.model import Model
print('The azureml.core version is {}'.format(azureml.core.VERSION))
The packages installed are:
absl-py==0.9.0
adal==1.2.7
ansiwrap==0.8.4
anyio==3.6.2
applicationinsights==0.11.10
argcomplete==1.12.3
argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
asn1crypto==1.3.0
astor==0.8.0
astunparse==1.6.3
attrs==22.1.0
azure-cli-core==2.43.0
azure-cli-telemetry==1.0.8
azure-common==1.1.28
azure-core==1.26.1
azure-graphrbac==0.61.1
azure-identity==1.7.0
azure-mgmt-authorization==3.0.0
azure-mgmt-compute==29.0.0
azure-mgmt-containerregistry==10.0.0
azure-mgmt-core==1.3.2
azure-mgmt-keyvault==10.1.0
azure-mgmt-resource==21.2.1
azure-mgmt-storage==20.1.0
azure-storage-blob==12.13.0
azure-storage-queue==12.5.0
azureml==0.2.7
azureml-automl-core==1.48.0
azureml-automl-runtime==1.48.0.post1
azureml-contrib-notebook==1.48.0
azureml-core==1.48.0
azureml-dataprep==4.8.3
azureml-dataprep-native==38.0.0
azureml-dataprep-rslex==2.15.1
azureml-dataset-runtime==1.48.0
azureml-explain-model==1.48.0
azureml-interpret==1.48.0
azureml-mlflow==1.48.0
azureml-pipeline==1.48.0
azureml-pipeline-core==1.48.0
azureml-pipeline-steps==1.48.0
azureml-sdk==1.48.0
azureml-telemetry==1.48.0
azureml-train-automl==1.48.0
azureml-train-automl-client==1.48.0
azureml-train-automl-runtime==1.48.0
azureml-train-core==1.48.0
azureml-train-restclients-hyperdrive==1.48.0
azureml-training-tabular==1.48.0
azureml-widgets==1.48.0
backcall==0.1.0
backports.tempfile==1.0
backports.weakref==1.0.post1
bcrypt==3.2.0
bleach==5.0.1
blinker==1.4
bokeh==2.4.3
boto==2.49.0
boto3==1.12.0
botocore==1.15.0
bpyutils==0.5.8
cachetools==4.1.1
certifi==2020.6.20
cffi==1.14.0
chardet==3.0.4
Click==7.0
cloudpickle==1.3.0
configparser==3.7.4
contextlib2==21.6.0
cryptography==2.8
cycler==0.10.0
Cython==0.29.15
dask==2021.6.0
databricks-cli==0.11.0
dataclasses==0.6
decorator==4.4.1
defusedxml==0.7.1
dill==0.3.1.1
diskcache==5.0.2
distributed==2021.6.0
distro==1.8.0
docker==4.3.1
docutils==0.15.2
dotnetcore2==3.1.23
entrypoints==0.3
fastjsonschema==2.16.2
fire==0.5.0
Flask==1.1.1
flatbuffers==22.12.6
fsspec==2022.11.0
fusepy==3.0.1
future==0.18.2
gast==0.3.3
gensim==3.8.3
gitdb==4.0.5
GitPython==3.1.0
google-auth==1.11.2
google-auth-oauthlib==0.4.1
google-pasta==0.2.0
grpcio==1.27.2
gunicorn==20.0.4
h5py==2.10.0
HeapDict==1.0.1
horovod==0.19.5
humanfriendly==10.0
idna==2.8
importlib-metadata==4.13.0
importlib-resources==5.10.1
interpret-community==0.27.0
interpret-core==0.2.7
ipykernel==5.1.4
ipython==7.12.0
ipython-genutils==0.2.0
ipywidgets==7.7.2
isodate==0.6.0
itsdangerous==1.1.0
jedi==0.14.1
jeepney==0.8.0
Jinja2==2.11.1
jmespath==0.10.0
joblib==0.14.1
joblibspark==0.2.0
jsonpickle==2.2.0
jsonschema==4.17.3
jupyter-client==5.3.4
jupyter-core==4.6.1
jupyter-server==1.23.3
jupyterlab-widgets==1.1.1
Keras-Preprocessing==1.1.2
keras2onnx==1.6.0
kiwisolver==1.1.0
knack==0.10.1
koalas==1.2.0
lightgbm==2.3.0
llvmlite==0.38.1
locket==1.0.0
Mako==1.1.2
Markdown==3.1.1
MarkupSafe==1.1.1
matplotlib==3.1.3
mistune==0.8.4
mkl-fft==1.0.15
mkl-random==1.1.0
mkl-service==2.3.0
ml-wrappers==0.2.2
mleap==0.16.1
mlflow==1.11.0
mlflow-databricks-artifacts==2.0.1
mlflow-skinny==1.30.0
msal==1.20.0
msal-extensions==1.0.0
msgpack==1.0.4
msrest==0.7.1
msrestazure==0.6.4
nbclassic==0.4.8
nbconvert==5.6.1
nbformat==5.7.0
ndg-httpsclient==0.5.1
nest-asyncio==1.5.6
networkx==2.4
nimbusml==1.8.0
nltk==3.4.5
notebook==6.5.2
notebook-shim==0.2.2
numba==0.55.2
numpy==1.18.1
oauthlib==3.1.0
olefile==0.46
onnx==1.12.0
onnxconverter-common==1.6.0
onnxmltools==1.4.1
onnxruntime==1.11.1
opt-einsum==3.3.0
packaging==21.3
pandas==1.1.5
pandocfilters==1.5.0
papermill==1.2.1
paramiko==2.7.1
parso==0.5.2
partd==1.3.0
pathspec==0.10.3
patsy==0.5.1
petastorm==0.9.5
pexpect==4.8.0
pickleshare==0.7.5
pipupgrade==1.12.0
pkginfo==1.9.2
pkgutil-resolve-name==1.3.10
plotly==4.9.0
pmdarima==1.7.1
portalocker==2.6.0
prometheus-client==0.15.0
prompt-toolkit==3.0.3
protobuf==3.20.1
psutil==5.9.4
psycopg2==2.8.6
ptyprocess==0.6.0
pyarrow==1.0.1
pyasn1==0.4.8
pyasn1-modules==0.2.7
pycparser==2.19
Pygments==2.5.2
PyJWT==2.6.0
PyNaCl==1.3.0
pyodbc===4.0.0-unsupported
pyOpenSSL==19.1.0
pyparsing==2.4.6
pyrsistent==0.19.2
PySocks==1.7.1
python-dateutil==2.8.1
python-editor==1.0.4
pytz==2019.3
PyYAML==5.3.1
pyzmq==18.1.1
querystring-parser==1.2.4
requests==2.22.0
requests-oauthlib==1.3.0
retrying==1.3.3
rsa==4.0
s3transfer==0.3.3
scikit-learn==0.22.2.post1
scipy==1.5.3
seaborn==0.10.0
SecretStorage==3.3.3
Send2Trash==1.8.0
shap==0.39.0
simplejson==3.17.0
six==1.14.0
skl2onnx==1.4.9
sklearn-pandas==1.7.0
slicer==0.0.7
smart-open==1.9.0
smmap==3.0.4
sniffio==1.3.0
sortedcontainers==2.4.0
spark-tensorflow-distributor==0.1.0
sqlparse==0.4.3
statsmodels==0.11.0
tabulate==0.8.3
tblib==1.7.0
tenacity==8.1.0
tensorboard==2.3.0
tensorboard-plugin-wit==1.7.0
tensorflow-cpu==2.3.0
tensorflow-estimator==2.3.0
termcolor==1.1.0
terminado==0.17.1
testpath==0.6.0
textwrap3==0.9.2
threadpoolctl==3.1.0
toolz==0.12.0
torch==1.6.0
torchvision==0.7.0
tornado==6.0.3
tqdm==4.42.1
traitlets==4.3.3
typing-extensions==4.4.0
urllib3==1.25.8
wcwidth==0.1.8
webencodings==0.5.1
websocket-client==0.56.0
Werkzeug==1.0.0
widgetsnbextension==3.6.1
wrapt==1.11.2
xgboost==1.1.1
zict==2.2.0
zipp==3.11.0
These packages are the ones installed by default on a ML single node cluster
12-20-2022 05:02 PM
Hi Kaniz, this approach didn't work and it's obvious. In the docs that you kindly provided me, they make an update of the azureml-sdk package for one highest than 1.42 but if you see my list of packages the version of mine is 1.48. Even so, I ran the installation they mentioned but it didn't work.
12-21-2022 08:05 AM
Hi @Kaniz Fatma, it failed. Just to be clear the VM is a Standard_DS3_v2 7.3 LTS ML with Apache Spark 3.0.1 and Scala 2.12. I'm trying just to reproduce an excercise of the BigBook of ML by Databricks.
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group