cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Need help with this python import error.

qyu
New Contributor II

I am using databricks runtime 9.1 LTS ML and I got this error when I tried to import Scikit Learn package. I got the following error message:

TypeError                                 Traceback (most recent call last)
<command-181041> in <module>
      1 # Scikit Learn ML Library
----> 2 from sklearn.metrics import *
      3 from sklearn.preprocessing import LabelEncoder
      4 from sklearn.preprocessing import MinMaxScaler, StandardScaler, PowerTransformer
      5 from sklearn.model_selection import train_test_split, TimeSeriesSplit
 
/databricks/python_shell/dbruntime/PythonPackageImportsInstrumentation/__init__.py in import_patch(name, globals, locals, fromlist, level)
    156             # Import the desired module. If you're seeing this while debugging a failed import,
    157             # look at preceding stack frames for relevant error information.
--> 158             original_result = python_builtin_import(name, globals, locals, fromlist, level)
    159 
    160             is_root_import = thread_local._nest_level == 1
 
/usr/lib/python3.8/importlib/_bootstrap.py in _find_and_load(name, import_)
 
/usr/lib/python3.8/importlib/_bootstrap.py in _find_and_load_unlocked(name, import_)
 
/usr/lib/python3.8/importlib/_bootstrap.py in _load_unlocked(spec)
 
/usr/lib/python3.8/importlib/_bootstrap.py in _load_backward_compatible(spec)
 
/databricks/python_shell/dbruntime/PostImportHook.py in load_module(self, fullname)
    214     def load_module(self, fullname):
    215         try:
--> 216             module = self.loader.load_module(fullname)
    217             notify_module_loaded(module)
    218         except (ImportError, AttributeError):
 
/databricks/python/lib/python3.8/site-packages/sklearn/__init__.py in <module>
     62 else:
     63     from . import __check_build
---> 64     from .base import clone
     65     from .utils._show_versions import show_versions
     66 
 
/databricks/python_shell/dbruntime/PythonPackageImportsInstrumentation/__init__.py in import_patch(name, globals, locals, fromlist, level)
    156             # Import the desired module. If you're seeing this while debugging a failed import,
    157             # look at preceding stack frames for relevant error information.
--> 158             original_result = python_builtin_import(name, globals, locals, fromlist, level)
    159 
    160             is_root_import = thread_local._nest_level == 1
.....
....
....
/databricks/python/lib/python3.8/site-packages/sklearn/externals/joblib/externals/cloudpickle/__init__.py in <module>
      1 from __future__ import absolute_import
      2 
----> 3 from .cloudpickle import *
      4 
      5 __version__ = '0.8.0'
 
/databricks/python_shell/dbruntime/PythonPackageImportsInstrumentation/__init__.py in import_patch(name, globals, locals, fromlist, level)
    156             # Import the desired module. If you're seeing this while debugging a failed import,
    157             # look at preceding stack frames for relevant error information.
--> 158             original_result = python_builtin_import(name, globals, locals, fromlist, level)
    159 
    160             is_root_import = thread_local._nest_level == 1
 
/databricks/python/lib/python3.8/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py in <module>
    150 
    151 
--> 152 _cell_set_template_code = _make_cell_set_template_code()
    153 
    154 
 
/databricks/python/lib/python3.8/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py in _make_cell_set_template_code()
    131         )
    132     else:
--> 133         return types.CodeType(
    134             co.co_argcount,
    135             co.co_kwonlyargcount,
 
TypeError: an integer is required (got type bytes)

1 ACCEPTED SOLUTION

Accepted Solutions

Atanu
Esteemed Contributor
Esteemed Contributor

Ok, if this library is included in this DBR , can you please check https://docs.databricks.com/release-notes/runtime/9.1ml.html . If not. please check if you are installing this library from cluster with all dependencies . Thanks.

View solution in original post

4 REPLIES 4

Kaniz
Community Manager
Community Manager

Hi @Qian Yu​ ! My name is Kaniz, and I'm the technical moderator here. Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question first. Or else I will get back to you soon. Thanks.

Atanu
Esteemed Contributor
Esteemed Contributor

@Qian Yu​  I believe the error is coming while you are executing any particular code on notebook.Could you please check the python version?

qyu
New Contributor II

@Atanu Sarkar​ I am using databricks runtime 9.1ML LTS and python version is 3.8.10

I am only just running import statement

from sklearn.metrics import *
from sklearn.preprocessing import LabelEncoder

Atanu
Esteemed Contributor
Esteemed Contributor

Ok, if this library is included in this DBR , can you please check https://docs.databricks.com/release-notes/runtime/9.1ml.html . If not. please check if you are installing this library from cluster with all dependencies . Thanks.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.