10-09-2023 01:57 AM
I'm just walking through a simple exercise presented in the Databricks Platform Lab notebook, in which I'm executing a remote notebook from within using the %run command. The remote notebook resides in the same directory as the Platform Lab notebook, and is named 'Notebook_B'. However, when I try to run it, a 'File not found' error is thrown. I've determined that the relative path to the notebook is correct. Might I be missing some extra bit of configuration? I've attached screenshots to illustrate.
Thanks for your time and assistance!
10-10-2023 01:26 AM
Hi @OliverCadman, Based on the information provided, it seems like you are trying to execute a notebook from within another notebook using the %run
command in Databricks.
However, you are encountering a File not found’ error even though the notebook you are trying to run (’ Notebook_B’) is in the same directory as the notebook from which you run the command.
The issue could be due to a few reasons:1. Incorrect notebook path: Even though you mentioned that the relative way is correct, it's worth double-checking. The path should be relative to the root directory of the Databricks workspace and should start with a /
. For example, if ’Notebook_B’ is in the same directory as the current notebook, you would run it as %run "./Notebook_B"
.
2. File extension: Include the correct file extension (.py, .scala, .sql, .r) in the notebook name. For example, if your notebook is a Python notebook, you should run it as %run "./Notebook_B.py"
.
3. Permissions: Ensure you have the necessary permissions to access and execute ’Notebook_B’.
Please review these points and try again.
10-10-2023 01:32 AM
Hi Kaniz,
Thank you for taking the time to respond to my query. I will take your advice on board and give it a shot. Much appreciated!
10-10-2023 10:35 PM
Hi @OliverCadman, We can build a thriving shared knowledge and insights community. Come back and mark the best answers to contribute to our ongoing pursuit of excellence.
10-17-2023 03:52 AM
I have stepped through your recommendations and I've not yet been able to successfully execute the %run command. I will check again and report back! Thanks
10-17-2023 03:58 AM
Hi @OliverCadman,
It looks like the error could be related to the working directory that is set when you run the command %run Notebook_B
. By default, Databricks sets the working directory to /databricks/driver/
, which might not be the directory that contains your Notebook_B
notebook.
To ensure that the working directory is set to the directory containing Notebook_B
, you can use the %cd
magic command to change the working directory before running %run
. Here's an example:
%cd /databricks/driver/my_notebook_directory/
%run Notebook_B
In this example, you should replace my_notebook_directory
with the actual directory that contains Notebook_B
.
Alternatively, you can specify an absolute path to Notebook_B
when you run %run
. Here's an example:
%run /dbfs/path/to/Notebook_B
In this example, you should replace /dbfs/path/to/Notebook_B
with the actual absolute path to Notebook_B
.
Finally, make sure that you have the necessary permissions to access the directory, especially if it is a shared directory. You can check the permissions by running the %fs ls
command and verify that you can see the directory containing Notebook_B
.
10-17-2023 04:49 AM
Thank you for taking the time to respond to my query.
10-17-2023 04:54 AM
Hi @aslaleratu , Please let us know if that works.
11-07-2023 06:47 AM
The %run command is a specific Jupyter magic command.
The ipykernel used in Databricks examines the initial line of code to determine the appropriate compiler or language for execution.
To minimize the likelihood of encountering errors, it is advisable to position the %run command as the first line in the cell and not have any command in the cell.
02-21-2024 03:12 AM
Hi,
I'm trying to run a notebook nested in a for loop in another notebook. %run command doesn't work in this case. Now I see the reason. Do you happen to know if there are other ways to run a notebook instead of using %run? Thank you.
07-14-2024 08:15 AM
This is exactly what I encountered. Even a # Comment line was giving me the same error. Removing the comment from that cell worked.
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group