'File not found' error when executing %run magic command

[OliverCadman]

I'm just walking through a simple exercise presented in the Databricks Platform Lab notebook, in which I'm executing a remote notebook from within using the %run command. The remote notebook resides in the same directory as the Platform Lab notebook, and is named 'Notebook_B'. However, when I try to run it, a 'File not found' error is thrown. I've determined that the relative path to the notebook is correct. Might I be missing some extra bit of configuration? I've attached screenshots to illustrate.

Thanks for your time and assistance!

[Kaniz_Fatma]

Hi @OliverCadman, Based on the information provided, it seems like you are trying to execute a notebook from within another notebook using the %run command in Databricks.

However, you are encountering a File not found’ error even though the notebook you are trying to run (’ Notebook_B’) is in the same directory as the notebook from which you run the command. 

The issue could be due to a few reasons:1. Incorrect notebook path: Even though you mentioned that the relative way is correct, it's worth double-checking. The path should be relative to the root directory of the Databricks workspace and should start with a /. For example, if ’Notebook_B’ is in the same directory as the current notebook, you would run it as %run "./Notebook_B".

2. File extension: Include the correct file extension (.py, .scala, .sql, .r) in the notebook name. For example, if your notebook is a Python notebook, you should run it as %run "./Notebook_B.py".

3. Permissions: Ensure you have the necessary permissions to access and execute ’Notebook_B’.

Please review these points and try again. 

[OliverCadman]

Hi Kaniz,

Thank you for taking the time to respond to my query. I will take your advice on board and give it a shot. Much appreciated!

[Kaniz_Fatma]

Hi @OliverCadman, We can build a thriving shared knowledge and insights community. Come back and mark the best answers to contribute to our ongoing pursuit of excellence.

[OliverCadman]

I have stepped through your recommendations and I've not yet been able to successfully execute the %run command. I will check again and report back! Thanks

[Kaniz_Fatma]

Hi @OliverCadman, 

It looks like the error could be related to the working directory that is set when you run the command %run Notebook_B. By default, Databricks sets the working directory to /databricks/driver/, which might not be the directory that contains your Notebook_B notebook.

To ensure that the working directory is set to the directory containing Notebook_B, you can use the %cd magic command to change the working directory before running %run. Here's an example:

%cd /databricks/driver/my_notebook_directory/
%run Notebook_B

In this example, you should replace my_notebook_directory with the actual directory that contains Notebook_B.

Alternatively, you can specify an absolute path to Notebook_B when you run %run. Here's an example:

%run /dbfs/path/to/Notebook_B

In this example, you should replace /dbfs/path/to/Notebook_B with the actual absolute path to Notebook_B.

Finally, make sure that you have the necessary permissions to access the directory, especially if it is a shared directory. You can check the permissions by running the %fs ls command and verify that you can see the directory containing Notebook_B.

[aslaleratu]

Thank you for taking the time to respond to my query.

[Kaniz_Fatma]

Hi @aslaleratu , Please let us know if that works.

[MuthuLakshmi]

The %run command is a specific Jupyter magic command.

The ipykernel used in Databricks examines the initial line of code to determine the appropriate compiler or language for execution.

To minimize the likelihood of encountering errors, it is advisable to position the %run command as the first line in the cell and not have any command in the cell.

[WoshiSeven]

Hi,

I'm trying to run a notebook nested in a for loop in another notebook. %run command doesn't work in this case. Now I see the reason. Do you happen to know if there are other ways to run a notebook instead of using %run? Thank you.

[BapsDBC]

This is exactly what I encountered. Even a # Comment line was giving me the same error. Removing the comment from that cell worked.