Reading single file from Databricks DBFS

Saf4Databricks
Contributor

I have a Test.csv file in FileStore of DBFS in Databricks Community edition. When I try to read the file using With Open, I get the following error:FileNotFoundError: [Errno 2] No such file or directory: '/dbfs/FileStore/tables/Test.csv'

 

import os
with open('/dbfs/FileStore/tables/Test.csv') as testFile:
    testContent = testFile.read()

 

Question: What could be a cause of the error and how can we fix it?

The file gets loaded successfully in a df as follows:

 

df = spark.read.csv("/FileStore/tables/Test.csv", header=True, inferSchema=True)