Python open function is unable to detect the file in dbfs

[billaspiel]

hi ,

Im a newbie learning spark using databricks , I did some investigation and searched if this questions was been asked earlier in community forum but unable to find anything so .

1. DBFS is unable to detect the file even though its present in it .

The issue happens only with below command

with open("dbfs:/FileStore/tables/data.txt") as f:

and not with

lines0 = sc.textFile("/FileStore/tables/data.txt"

Does this mean in databricks notebook we can't use python open function to open a file ?

[mathan_pillai · Accepted solution]

Hi @billaspiel

Can you please try by removing the colon ":" ? Here is a documentation on how to read/write the data

https://docs.databricks.com/user-guide/dbfs-databricks-file-system.html#access-dbfs-using-local-file...

If the issue still occurs, please share the complete statement and also the error message you are getting. Let us know how it goes.

Thanks

[billaspiel]

Hi mathan,

Thanks for the information , in addition to removing colon':' added '/' as well before dbfs which resolved the issue .

working syntax

----------------------

with open("/dbfs/FileStore/tables/data.txt") as f:

[Dflo]

I am having similar issues currently. I can read or access my storage account but when I attempted to read or access the container it told me path not found. I create the container and have full access as an owner.