12-15-2022 02:56 PM
I have stored a test.py in the dbfs at the below location "/dbfs/FileStore/shared_uploads/krishna@company.com/Project_Folder/test.py"
I have a print statement in test.py which says the below
print( os.getcwd() )
and it prints the below
'/databricks/driver'
This is utterly weird as I am not getting the correct path
Btw, the same thing happens with a notebook as well.
So I have a notebook "test_nb.ipynb"" at the below location
"/dbfs/FileStore/shared_uploads/krishna@company.com/Project_Folder/test_nb.ipynb"
I have the same print statement in test_nb.py which says the below:
print( os.getcwd() )
and it prints the below
'/databricks/driver'
Can someone please help me to print the correct path ?? When you answer, please can you make sure not to make the answer specific to a notebook as I actually need to print the path of a .py file
TIA
12-15-2022 06:11 PM
You can get the notebook path using following code
dbutils.notebook.\
entry_point.getDbutils().\
notebook().\
getContext().\
notebookPath().\
value()
Or, you can use Workspace APIs to list notebooks.
12-15-2022 09:39 PM
No bro.. this is just specifically for notebooks... Do you know something for .py files?
Moreover the code above doesn't even work for notebooks .. The answer I got with your code is below
/Users/krishna@company.com/Project_Folder/test_nb.ipynb
while I expected the path to be where I created the notebook.. (below)
"/dbfs/FileStore/shared_uploads/krishna@company.com/Project_Folder/test_nb.ipynb"
12-15-2022 10:44 PM
You can use a recursive function to list everything within a directory path, and then add a filter for .py files.
def get_dir_content(ls_path):
for dir_path in dbutils.fs.ls(ls_path):
if dir_path.isFile():
yield dir_path.path
elif dir_path.isDir() and ls_path != dir_path.path:
yield from get_dir_content(dir_path.path)
list(get_dir_content('dbfs:/FileStore'))
Source: stackvoerflow
12-16-2022 07:44 AM
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
Click here to register and join today!
Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.