cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

FileNotFoundError: [Errno 2] No such file or directory: '/FileStore/tables/flight_data.zip' The data and file exists in location mentioned above

Data_Engineer_3
New Contributor III

I am new to learning Spark and working on some practice; I have uploaded a zip file in DBFS /FileStore/tables directory and trying to run a python code to unzip the file; The python code is as: 

from zipfile import *

with ZipFile("/FileStore/tables/flight_data.zip", "r") as zipObj:

zipObj.extractall()

It throws an error:

FileNotFoundError: [Errno 2] No such file or directory: '/FileStore/tables/flight_data.zip'

When i check manually and also through the code dbutils.fs.ls("/FileStore/tables/") it returns

Out[13]: [ FileInfo(path='dbfs:/FileStore/tables/flight_data.zip', name='flight_data.zip', size=59082358)]

Can someone please review and advise; I am using community edition to run this on cluster with configuration:

Data Bricks Runtime Version 8.3 (includes Apache Spark 3.1.1, Scala 2.12)

1 ACCEPTED SOLUTION

Accepted Solutions

Kaniz_Fatma
Community Manager
Community Manager

Hi @Goutam Pal​ , Can you please tell me the Databricks Runtime Version on which you're trying this?

On community edition, in DBR 7+, this mount is disabled.

Please try the same on any DBR version less than 7. It'll work.

Thanks.

View solution in original post

17 REPLIES 17

Kaniz_Fatma
Community Manager
Community Manager

Hi @ Data_Engineer_241188! My name is Kaniz, and I'm the technical moderator here. Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question first. Or else I will get back to you soon. Thanks.

Hubert-Dudek
Esteemed Contributor III

It is on dbfs mount so in most scenarios you should prefix everything with /dbfs (or dbfs:/ in databricks native functions, in many is not even needed as they handle only dbfs like dbutils). So please try:

from zipfile import *
 
with ZipFile("/dbfs/FileStore/tables/flight_data.zip", "r") as zipObj:
    zipObj.extractall()

​@Hubert Dudek​ 

Hello Sir,

I tried in the way you suggested as well.However no luck! Still gives the same error.

Thank you,

Goutam​

Hubert-Dudek
Esteemed Contributor III

Do you have maybe high-concurrency server or some limited trial version (trial/free can make problem with reading with not native libraries).

Try also to explorer filesystem using shell commands by putting magic %sh in the first line in notebook to see is there /dbfs folder

%sh
ls /

@Hubert Dudek​ 

Hi Sir,

Working in community edition; Tried with magic commands as well.No luck! It says the command is not recognized.

Hubert-Dudek
Esteemed Contributor III

so it seems that in community edition you can not direct access filesystem. You have access only to dbfs storage but you need to load there uncompressed object. So you need everywhere to prefix with dbfs:/ if it is not work for some function it will not work. As a last chance you can give a try like that:

    from zipfile import *
     
    with ZipFile("dbfs:/FileStore/tables/flight_data.zip", "r") as zipObj:
        zipObj.extractall()

@Hubert Dudek​ 

Hi Sir, No luck with this way also. :(.​

Thank you for all the great suggestions though.😊

Hi @Goutam Pal​ ,

Please try the below code:-

from zipfile import ZipFile
file="/dbfs/FileStore/tables/iris.zip"
with ZipFile(file,'r')as zip:
  zip.printdir()
  print('processing........')
  zip.extractall()
  print("Process completed")

Screenshot 2021-11-11 at 4.16.12 PM

Hi @Goutam Pal​ ,

Are you still having this issue? I think @Kaniz Fatma​  example will work great to solve your issue.

Data_Engineer_3
New Contributor III

@Jose Gonzalez​ @Kaniz Fatma​ : The issue still persists. Please find attached the screenshot of the error.

Thanks,

Goutam Pal

Anonymous
Not applicable

@Goutam Pal​ - Thank you for letting us know. I apologize about the inconvenience.

Kaniz_Fatma
Community Manager
Community Manager

Hi @Goutam Pal​ , Can you please tell me the Databricks Runtime Version on which you're trying this?

On community edition, in DBR 7+, this mount is disabled.

Please try the same on any DBR version less than 7. It'll work.

Thanks.

Hello Kaniz..Will try and revert you back.

Hi @Goutam Pal​ , Sure. Please let us know if it worked for you.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group