cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

'File not found' error when executing %run magic command

OliverCadman
New Contributor III

I'm just walking through a simple exercise presented in the Databricks Platform Lab notebook, in which I'm executing a remote notebook from within using the %run command. The remote notebook resides in the same directory as the Platform Lab notebook, and is named 'Notebook_B'. However, when I try to run it, a 'File not found' error is thrown. I've determined that the relative path to the notebook is correct. Might I be missing some extra bit of configuration? I've attached screenshots to illustrate.

Thanks for your time and assistance!

 

 

10 REPLIES 10

Kaniz_Fatma
Community Manager
Community Manager

Hi @OliverCadmanBased on the information provided, it seems like you are trying to execute a notebook from within another notebook using the %run command in Databricks.

However, you are encountering a File not found’ error even though the notebook you are trying to run (’ Notebook_B’) is in the same directory as the notebook from which you run the command. 

The issue could be due to a few reasons:1. Incorrect notebook path: Even though you mentioned that the relative way is correct, it's worth double-checking. The path should be relative to the root directory of the Databricks workspace and should start with a /. For example, if ’Notebook_B’ is in the same directory as the current notebook, you would run it as %run "./Notebook_B".

2. File extension: Include the correct file extension (.py, .scala, .sql, .r) in the notebook name. For example, if your notebook is a Python notebook, you should run it as %run "./Notebook_B.py".

3. Permissions: Ensure you have the necessary permissions to access and execute ’Notebook_B’.

Please review these points and try again. 

OliverCadman
New Contributor III

Hi Kaniz,

Thank you for taking the time to respond to my query. I will take your advice on board and give it a shot. Much appreciated!

Hi @OliverCadman, We can build a thriving shared knowledge and insights community. Come back and mark the best answers to contribute to our ongoing pursuit of excellence.

OliverCadman
New Contributor III

I have stepped through your recommendations and I've not yet been able to successfully execute the %run command. I will check again and report back! Thanks

Kaniz_Fatma
Community Manager
Community Manager

Hi @OliverCadman

It looks like the error could be related to the working directory that is set when you run the command %run Notebook_B. By default, Databricks sets the working directory to /databricks/driver/, which might not be the directory that contains your Notebook_B notebook.

To ensure that the working directory is set to the directory containing Notebook_B, you can use the %cd magic command to change the working directory before running %run. Here's an example:

%cd /databricks/driver/my_notebook_directory/
%run Notebook_B

In this example, you should replace my_notebook_directory with the actual directory that contains Notebook_B.

Alternatively, you can specify an absolute path to Notebook_B when you run %run. Here's an example:

%run /dbfs/path/to/Notebook_B

In this example, you should replace /dbfs/path/to/Notebook_B with the actual absolute path to Notebook_B.

Finally, make sure that you have the necessary permissions to access the directory, especially if it is a shared directory. You can check the permissions by running the %fs ls command and verify that you can see the directory containing Notebook_B.

 

aslaleratu
New Contributor II

Thank you for taking the time to respond to my query.

Hi @aslaleratu , Please let us know if that works.

 

MuthuLakshmi
New Contributor III

The %run command is a specific Jupyter magic command.

The ipykernel used in Databricks examines the initial line of code to determine the appropriate compiler or language for execution.

To minimize the likelihood of encountering errors, it is advisable to position the %run command as the first line in the cell and not have any command in the cell.

Hi,

I'm trying to run a notebook nested in a for loop in another notebook. %run command doesn't work in this case. Now I see the reason. Do you happen to know if there are other ways to run a notebook instead of using %run? Thank you.

BapsDBC
New Contributor III

This is exactly what I encountered. Even a # Comment line was giving me the same error. Removing the comment from that cell worked.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group