topic Re: Databricks Magic Command in Data Engineering

Databricks Magic Command

Wolverine — Tue, 02 Jul 2024 20:32:54 GMT

I am trying few commands

what is the equivalent magic command of

dbutils.fs.rm("dbfs:/sampledir",True)

Actually I am looking how to use magic commands in same way as dbutils . For Instance

dbutils.fs.head('dbfs:/FileStore/<<name>>.csv',10) Gives 10 bytes I am looking for magic command like how to give second parameter %fs head dbfs:/FileStore/hotel_data.csv 10(If we give upto 10) it works but when we pass second one like 10 it will not give output like 10 bytes as above

Re: Databricks Magic Command

Giri-Patcham — Tue, 02 Jul 2024 20:23:58 GMT

@Wolverine Try %fs rm -r sampledir
to delete recursively

Re: Databricks Magic Command

Witold — Tue, 02 Jul 2024 20:26:28 GMT

You could use shell commands, like

%sh rm -r sampledir

You need to check for the correct path before, I currently don't know where dbfs folders are exactly mounted

Re: Databricks Magic Command

Wolverine — Tue, 02 Jul 2024 20:30:49 GMT

Ok Actually I am looking how to use magic commands in same way as dbutils . For Instance