Error importing pip package s3fs

User16753724663
Databricks Employee
Databricks Employee

‎06-16-2021 10:11 AM

A job recently began failing with the following error when a python notebook imports the pip package s3fs.

ImportError: cannot import name 'maybe_sync' from 'fsspec.asyn' (/databricks/python/lib/python3.8/site-packages/fsspec/asyn.py)
 
ImportError Traceback (most recent call last)
 
in
 
11 import time
 
12 from pathlib import Path
 
---> 13 import s3fs
 
14
 
15 pd.options.display.max_rows = 1000
 
 
 
/databricks/python/lib/python3.8/site-packages/s3fs/__init__.py in
 
----> 1 from .core import S3FileSystem, S3File
 
2 from .mapping import S3Map
 
3
 
4 from ._version import get_versions
 
5
 
 
 
/databricks/python/lib/python3.8/site-packages/s3fs/core.py in
 
9 from fsspec.spec import AbstractBufferedFile
 
10 from fsspec.utils import infer_storage_options, tokenize
 
---> 11 from fsspec.asyn import AsyncFileSystem, sync, sync_wrapper, maybe_sync
 
12
 
13 import aiobotocore

Labels:
0 Kudos

User16753724663
Databricks Employee
Databricks Employee

‎06-16-2021 10:12 AM

While checking the init script is installing the s3fs version 0.5.2.

This version has issues at the moment from the pypi. I have tested version 0.6.0 that works fine.

please change your requirement.txt file with a newer version of s3fs. Below is the path where the requirement.txt is residing

/dbfs/scripts/oce/requirements.txt

After changing the version to 0.6.0, we will no longer get the import error.

If you would like to try installing manually the pypi library below command will help you :

pip install s3fs==0.6.0

0 Kudos