Databricks

User16753724663 · ‎06-16-2021

I have an issue to install and use kneed python libary.

I can install it and check it from log.

[Install command]

%sh

pip install kneed

[log]

Installing collected packages: kneed

Successfully installed kneed-0.7.0

but when I call this library it shows error:

ModuleNotFoundError: No module named 'kneed'

I got this error after changing cluster to "5.5 LTS ML (includes Apache Spark 2.4.3, Scala 2.11)"

User16753724663 · ‎06-16-2021

The kneed library has a dependency and we need to install them as well in order to work:

numpy==1.18

scipy==1.1.0

scikit-learn==0.21.3

Once we install the above libraries using GUI, we can run the below command to check the installed library with the correct version:

%sh
/databricks/python3/bin/pip3 list

Since the DBR 5.5 has both python version ie python2 and python3, hence we need to use the above command.

Please use the specific versions only.

Unable to install kneed library in cluster with DBR version 5.5 LTS