Copy files and folder into Users area , but Files are not showing in UI

[rkshanmugaraja]

Hi All

I'm trying to copy the whole training directory (which contains multiple sub folders and files) from my catalog volume area to each users area. 

From : "dbfs:/Volumes/CatalogName/schema/Training"

To : "dbfs:/Workspace/Users/username@domain.com/Training/"

I tried using

1) dbutils.fs.cp(sourcePath , destPath, recursive=true)

Its says it successfully copied but inside the UI its not visible , If I run the ls command , I can see the files and folders. 

2) I also tried using the put API /api/2.0/dbfs/put 

its the same it says it successful but the files are not visible in the UI. 

Is there a way other than UI, because need to copy the folders to 50 users area ? Can't do it manually. 

[radothede]

Hi  ,

Do You want to use dbfs location on purpose, or You want to upload the training notebooks to Workspace/users location? The reason I'm asking is those are two different locations, although both are related to file management in Databricks. (see: https://docs.databricks.com/en/files/index.html)

If You want to import directory to Workspace/users, one way to go would be using databricks cli.

It is a simple rest api wrapper, so it is easier to handle.

Steps would be to:

• Install databricks cli – You can use legacy version using pip install databricks-cli or download latest version using manual provided here: https://docs.databricks.com/en/dev-tools/cli/install.html
• Configure the connection settings using databricks configure – easiest way would be to provide host and access token
• Use databricks workspace import_dir command to import directory from Your local maschine to Workspace/users locations.

Hope that helps 😊

Best,

Rado.