cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

How to print the path of a .py file or a notebook?

KrishZ
Contributor

I have stored a test.py in the dbfs at the below location "/dbfs/FileStore/shared_uploads/krishna@company.com/Project_Folder/test.py"

I have a print statement in test.py which says the below

print( os.getcwd() )

and it prints the below

'/databricks/driver'

This is utterly weird as I am not getting the correct path

Btw, the same thing happens with a notebook as well.

So I have a notebook "test_nb.ipynb"" at the below location

"/dbfs/FileStore/shared_uploads/krishna@company.com/Project_Folder/test_nb.ipynb"

I have the same print statement in test_nb.py which says the below:

print( os.getcwd() )

and it prints the below

'/databricks/driver'

 Can someone please help me to print the correct path ?? When you answer, please can you make sure not to make the answer specific to a notebook as I actually need to print the path of a .py file

TIA

4 REPLIES 4

AmanSehgal
Honored Contributor III

You can get the notebook path using following code

dbutils.notebook.\
entry_point.getDbutils().\
notebook().\
getContext().\
notebookPath().\
value()

 Or, you can use Workspace APIs to list notebooks.

No bro.. this is just specifically for notebooks... Do you know something for .py files?

Moreover the code above doesn't even work for notebooks .. The answer I got with your code is below

/Users/krishna@company.com/Project_Folder/test_nb.ipynb

while I expected the path to be where I created the notebook.. (below)

"/dbfs/FileStore/shared_uploads/krishna@company.com/Project_Folder/test_nb.ipynb"

AmanSehgal
Honored Contributor III

You can use a recursive function to list everything within a directory path, and then add a filter for .py files.

def get_dir_content(ls_path):
    for dir_path in dbutils.fs.ls(ls_path):
        if dir_path.isFile():
            yield dir_path.path
        elif dir_path.isDir() and ls_path != dir_path.path:
            yield from get_dir_content(dir_path.path)
    
list(get_dir_content('dbfs:/FileStore'))

Source: stackvoerflow

Aviral-Bhardwaj
Esteemed Contributor III

Hey @Krishna Zanwarโ€‹  Please use the below code this will work

image 

and as you want the specific location you can create a custom code and format the path using a python formatter , it will give you desired result .

AviralBhardwaj

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group