Cedric
Databricks Employee
Databricks Employee

Hi Thiam,

Thank you for reaching out to us. In this case it seems that you have written a file to the OS /tmp and tried to fetch the same folder in DBFS.

Written >> /tmp/TotalData.csv

Reading >> /dbfs/tmp/TotalData.csv

Please try to execute write.csv with the following path instead:

write.csv(TotalData,file='/dbfs/tmp/TotalData.csv',row.names = FALSE)

%fs ls /tmp

Then you will see the file in DBFS. If your intention was to write it to the OS /tmp, please do the following:

write.csv(TotalData,file='/tmp/TotalData.csv',row.names = FALSE)

%sh ls /tmp

%sh allows you to run shell code on the OS while %fs is used for DBFS file access.

Please read the following documentation for more information:

https://docs.databricks.com/notebooks/notebooks-use.html#:~:text=Notebooks%20also%20support%20a%20fe...