topic Re: /dbfs is empty in Data Engineering

/dbfs is empty

FemiAnthony — Fri, 05 Nov 2021 09:39:42 GMT

Why does /dbfs seem to be empty in my Databricks cluster ?

If I run

%sh ls /dbfs

I get no output.

I am looking for the databricks-datasets subdirectory ? I can't find it under /dbfs

Re: /dbfs is empty

Sajesh — Fri, 05 Nov 2021 09:50:51 GMT

Hi @Femi Anthony , Do you see any results on executing: display(dbutils.fs.ls("/"))?

What's the DBR version of the cluster?

Re: /dbfs is empty

Prabakar — Fri, 05 Nov 2021 09:53:37 GMT

Hi @Femi Anthony Please let us know if you are trying this in Community edition or in your workspace?

If it's CE then you won't be able to see anything under /dbfs while using %sh.

display(dbutils.fs.ls("/")) should show the dataset folder.

Re: /dbfs is empty

FemiAnthony — Fri, 05 Nov 2021 14:12:39 GMT

Yes, this is in Community ediition. Thanks.

Re: /dbfs is empty

Prabakar — Fri, 05 Nov 2021 14:34:24 GMT

Okay. Then this is the limitation with CE.

Use display(dbutils.fs.ls("/"))

Re: /dbfs is empty

jose_gonzalez — Mon, 08 Nov 2021 20:52:57 GMT

Hi @Femi Anthony ,

If @Prabakar Ammeappin's response helped you to solve your question? would you be happy to mark his answer as best so that others can quickly find the solution in the future

Re: /dbfs is empty

FemiAnthony — Tue, 09 Nov 2021 09:09:39 GMT

Thanks @Prabakar Ammeappin