topic How to print the path of a .py file or a notebook? in Data Engineering

How to print the path of a .py file or a notebook?

KrishZ — Thu, 15 Dec 2022 22:56:57 GMT

I have stored a test.py in the dbfs at the below location "/dbfs/FileStore/shared_uploads/krishna@company.com/Project_Folder/test.py"

I have a print statement in test.py which says the below

print( os.getcwd() )

and it prints the below

'/databricks/driver'

This is utterly weird as I am not getting the correct path

Btw, the same thing happens with a notebook as well.

So I have a notebook "test_nb.ipynb"" at the below location

"/dbfs/FileStore/shared_uploads/krishna@company.com/Project_Folder/test_nb.ipynb"

I have the same print statement in test_nb.py which says the below:

print( os.getcwd() )

and it prints the below

'/databricks/driver'

Can someone please help me to print the correct path ?? When you answer, please can you make sure not to make the answer specific to a notebook as I actually need to print the path of a .py file

TIA

Re: How to print the path of a .py file or a notebook?

AmanSehgal — Fri, 16 Dec 2022 02:11:00 GMT

You can get the notebook path using following code

dbutils.notebook.\
entry_point.getDbutils().\
notebook().\
getContext().\
notebookPath().\
value()

Or, you can use Workspace APIs to list notebooks.

Re: How to print the path of a .py file or a notebook?

KrishZ — Fri, 16 Dec 2022 05:39:38 GMT

No bro.. this is just specifically for notebooks... Do you know something for .py files?

Moreover the code above doesn't even work for notebooks .. The answer I got with your code is below

/Users/krishna@company.com/Project_Folder/test_nb.ipynb

while I expected the path to be where I created the notebook.. (below)

"/dbfs/FileStore/shared_uploads/krishna@company.com/Project_Folder/test_nb.ipynb"

Re: How to print the path of a .py file or a notebook?

AmanSehgal — Fri, 16 Dec 2022 06:44:37 GMT

You can use a recursive function to list everything within a directory path, and then add a filter for .py files.

def get_dir_content(ls_path):
    for dir_path in dbutils.fs.ls(ls_path):
        if dir_path.isFile():
            yield dir_path.path
        elif dir_path.isDir() and ls_path != dir_path.path:
            yield from get_dir_content(dir_path.path)
    
list(get_dir_content('dbfs:/FileStore'))

Source: stackvoerflow

Re: How to print the path of a .py file or a notebook?

Aviral-Bhardwaj — Fri, 16 Dec 2022 15:44:50 GMT

Hey @Krishna Zanwar Please use the below code this will work

and as you want the specific location you can create a custom code and format the path using a python formatter , it will give you desired result .