Not able to move files from local to dbfs through dbfs CLI

[study_community]

Hi Folks,

I have installed and configured databricks CLI in my local machine. I tried to move a local file from my personal computer using dbfs cp to dbfs:/ path. I can see the file is copied from local, and is only visible in local. I am not able to see it in dbfs cluster via databricks UI.

 

Local

I'm trying to list the data in cluster. It is not available .

 

[Hubert-Dudek]

for cli command is "databricks fs cp" to copy from local storage to cluster

[study_community]

I tried databricks fs cp too @Hubert Dudek​  , still the issue exists. Any Idea why?

[Hubert-Dudek]

What distribution are you using (community, Azur?)

can you verify your connection using databricks fs ls

do you see there typical folders like user, tmp etc?

[study_community]

Hi @Hubert Dudek​  , I am using community edition with AWS in the backend.

If I do databricks fs ls, I only am able to see one folder, databricks-results

 

[study_community]

Okay Sure. I'll use upload then

[Kaniz_Fatma]

Hi @Thulasitharan Govindaraj​ , Does the issue persist or is it resolved?

[study_community]

Issue still exists @Kaniz Fatma​ 

[Kaniz_Fatma · Accepted solution]

Hi @Thulasitharan Govindaraj​ ,

The DBFS command-line interface (CLI) uses the DBFS API to expose an easy to use the command-line interface to DBFS. Using this client, you can interact with DBFS using commands similar to those you use on a Unix command line. For example:

# List files in DBFS
dbfs ls
# Put local file ./apple.txt to dbfs:/apple.txt
dbfs cp ./apple.txt dbfs:/apple.txt
# Get dbfs:/apple.txt and save to local file ./apple.txt
dbfs cp dbfs:/apple.txt ./apple.txt
# Recursively put local dir ./banana to dbfs:/banana
dbfs cp -r ./banana dbfs:/banana

Reference

[Kaniz_Fatma]

Hi @Thulasitharan Govindaraj​ , Were you able to resolve your problem through the solution I provided above?

[study_community]

No, @Kaniz. The syntax I was using is the same as you have mentioned. I still am not able to see the file which I copied from local to dbfs in databricks front-end

[Kaniz_Fatma]

Hi @Thulasitharan Govindaraj​ , Did you install and configure Azure Databricks CLI properly? Would you mind trying again?

[Kaniz_Fatma]

Hi @Thulasitharan Govindaraj​ , What is the size of the file which you're trying to copy from the local to dbfs?

[Anonymous]

Hi, Could you try to save the file from your local machine to dbfs:/FileStore location?

# Put local file test.py to dbfs:/FileStore/test.py

dbfs cp test.py dbfs:/FileStore/test.py