File Not Found Error while reading pickle file

[divyasri1504]

Hello, there

I have a pickle file uploaded in a mounted location in databricks ( /dbfs/mnt/blob/test.pkl). I am trying to read this pickle file using the below python snippet

with open(path + "test.pkl", "rb") as f:
       bands = pickle.load(f)

But it throws FileNotFoundError: [Errno 2] No such file or directory: ' /dbfs/mnt/blob/test.pkl'. But the file is present in that location. I have tried listing using dbutils.fs.ls as well and file is present. Can anyone help me with this issue.

Thanks

[Kaniz_Fatma]

Hi @divyasri1504 ,

• Make sure you’re using the correct path to access the file. In Databricks, you should typically prefix everything with /dbfs (or dbfs:/ for native functions). Try using the full path like this:

with open("/dbfs/mnt/blob/test.pkl", "rb") as f:
    bands = pickle.load(f)

• If you’re using the community edition, in Databricks Runtime (DBR) 7+, certain mounts are disabled. Try running your code on a DBR version less than 7.
• Ensure that the file has the correct permissions. You can use dbutils.fs.ls("/dbfs/mnt/blob/") to verify that the file is listed.
• Also, check if there are any restrictions on reading files from mounted locations.
• If the issue persists, consider using dbutils.fs.cp to copy the file from DBFS to a local directory (e.g., /tmp) and then read it from there:

  dbutils.fs.cp("/dbfs/mnt/blob/test.pkl", "file:///tmp/test.pkl")
  with open("/tmp/test.pkl", "rb") as f:
      bands = pickle.load(f)
  

Hopefully, one of these approaches will help you resolve the issue! 😊

If you need further assistance, feel free to ask!