How to move files of same extension in databricks files system?

bkr — Fri, 08 Jun 2018 13:29:45 GMT

I am facing file not found exception when i am trying to move the file with * in DBFS. Here both source and destination directories are in DBFS. I have the source file named "test_sample.csv" available in dbfs directory and i am using the command like below from notebook cell,

dbutils.fs.mv("dbfs:/usr/krishna/sample/test*.csv", "dbfs:/user/abc/Test/Test.csv")

Error:

java.io.FileNotFoundException: dbfs:/usr/krishna/sample/test*.csv

Please suggest if there is any other way to do this. Thanks.

Re: How to move files of same extension in databricks files system?

ricardo_portill — Fri, 08 Jun 2018 16:45:33 GMT

@bkr, you can reference the file name using dbutils and then pass this to the move command. Here's an example for this in Scala:

val fileNm = dbutils.fs.ls("/usr/krishna/sample").map(_.name).filter(r => r.startsWith("test"))(0)
val fileLoc = "dbfs:/usr/krishna/sample/sample/" + fileNm
dbutils.fs.mv(fileLoc, "dbfs:/user/abc/Test/Test.csv")

topic How to move files of same extension in databricks files system? in Data Engineering

How to move files of same extension in databricks files system?

Re: How to move files of same extension in databricks files system?