โ01-05-2022 02:19 AM
I try to import **hdbscan** but it throws this following error
/databricks/python_shell/dbruntime/PythonPackageImportsInstrumentation/__init__.py in import_patch(name, globals, locals, fromlist, level)
156 # Import the desired module. If youโre seeing this while debugging a failed import,
157 # look at preceding stack frames for relevant error information.
--> 158 original_result = python_builtin_import(name, globals, locals, fromlist, level)
159
160 is_root_import = thread_local._nest_level == 1
hdbscan/_hdbscan_linkage.pyx in init hdbscan._hdbscan_linkage()
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 80 from PyObject
When I update the numpy version to 1.22.0, I am not getting this error, but it fails to import **umap** which tells the numpy version to be <1.20.
To summarize
Databricks runtime version - 10.1 ML (includes Apache Spark 3.2.0, Scala 2.12)
Python Version - 3.8.10
Python packages installed
umap-learn==0.5.1
numpy==1.22.0
hdbscan==0.8.27
(This version throws error while importing umap)
Python packages installed
umap-learn==0.5.1
numpy==1.20.0
hdbscan==0.8.27
(This version throws the above error)
โ01-30-2022 11:27 PM
Hi @Rajamannar Aanjaramโ , It looks like there's a compatibility issue with the
hdbscan library.
You may check out the Github issue which addresses a similar issue.
In case the above Github issue doesn't solve your issue, I would request to open a new issue here: https://github.com/scikit-learn-contrib/hdbscan/issues
Hope this will help. Please let us know if any further queries.
โ01-06-2022 04:03 AM
Hi @ Itachi_Naruto! My name is Kaniz, and I'm the technical moderator here. Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question first. Or else I will get back to you soon. Thanks.
โ01-06-2022 04:11 AM
Hi @Rajamannar Aanjaramโ , Please try these commands, You'll be able to install hdbscan package.
pip install --upgrade numpy
pip install hdbscan
โ01-06-2022 07:30 AM
Hi @Kaniz Fatmaโ โ thanks for the answer, but when we run the above commands I can't import the umap library
โ01-10-2022 07:27 AM
Hi @Rajamannar Aanjaramโ ,
To use UMAP you need to install umap-learn not umap.
So, in case you've installed umap please run the following commands to uninstall umap and install upam-learn instead:
pip uninstall umap
pip install umap-learn
And then in your python code make sure you are importing the module using:
import umap.umap_ as umap
Instead of
import umap
โ01-11-2022 08:26 PM
does this help @Rajamannar Aanjaramโ ?
โ01-12-2022 12:05 AM
Hi @Atanu Sarkarโ , no this solution didn't work
โ01-30-2022 11:27 PM
Hi @Rajamannar Aanjaramโ , It looks like there's a compatibility issue with the
hdbscan library.
You may check out the Github issue which addresses a similar issue.
In case the above Github issue doesn't solve your issue, I would request to open a new issue here: https://github.com/scikit-learn-contrib/hdbscan/issues
Hope this will help. Please let us know if any further queries.
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโt want to miss the chance to attend and share knowledge.
If there isnโt a group near you, start one and help create a community that brings people together.
Request a New Group