Cannot import editable installed module in notebook

[newenglander]

Hi,

I have the following directory structure:

- mypkg/
  - setup.py
  - mypkg/
    - __init__.py
    - module.py
- scripts/
  - main # notebook
  

From the `main` notebok I have a cell that runs:

%pip install -e /path/to/mypkg

This command appears to successfully install the package. And in a subsequent cell I can run:

!python -m mypkg.module

and it executes successfully.

However, when I try in a notebook cell to simply run:

import mypkg.module

the execution fails with the message: 

ModuleNotFoundError: No module named 'mypkg'

I've tried executing the import both before and after running `%restart_python` and it fails either way.

Please advise.

Thank you

[Louis_Frolio]

Hey @newenglander — always great to meet a fellow New Englander 😊

Could you share a bit more detail about your setup? For example, are you running on classic compute or serverless? And are you working in a customer workspace, or using Databricks Free Edition (formerly Community Edition)?

Any additional context you can provide will help me point you in the right direction.

Cheers, Louis.

[mark_ott]

This issue happens because Jupyter uses a different Python environment from the one where you installed mypkg in editable mode. Although running
%pip install -e /path/to/mypkg inside the notebook prints a success message, that command may run in a separate interpreter than the one your active notebook kernel uses.

When you later execute !python -m mypkg.module, it works only because !python launches an external process that uses your system or virtualenv Python — not the running Jupyter kernel’s Python session.​

Confirm what’s happening

To verify the mismatch:

python

import sys print(sys.executable) !which python !jupyter kernelspec list

If these point to different paths, that’s the cause of your ModuleNotFoundError.

Solutions

1. Ensure your notebook kernel uses the same environment

Activate the same environment you used for the editable install and register it as a Jupyter kernel:

bash

source /path/to/venv/bin/activate pip install ipykernel python -m ipykernel install --user --name mypkg-env --display-name "mypkg env"

Then reopen your notebook and select “mypkg env” from the kernel list.​

2. Verify installation path

In a notebook cell, print out:

python

import site site.getsitepackages()

Check that the editable-path .egg-link file for mypkg is in one of these directories.

3. As a fallback, add it to sys.path manually

If switching kernels is not possible, append the install location:

python

import sys sys.path.append("/path/to/mypkg") import mypkg.module

This mirrors what happens automatically in the terminal’s sys.path but not in Jupyter’s environment.​

4. Optionally reinstall inside the notebook’s environment

If you confirm that the notebook uses a different environment (e.g., Conda base or system Python), reinstall mypkg inside it:

python

%pip install -e /path/to/mypkg --force-reinstall

Once the kernel and Jupyter runtime point to the same Python interpreter where the editable installation resides, import mypkg.module will succeed consistently.

[newenglander]

@mark_ott and @Louis_Frolio  thanks for the responses.

Louis -- I am using classic compute on a customer workspace. If it matters I am on DBR 15.4 ML.

Matt -- It appears to be as you said, that Jupyter is using a different python environment.

`sys.executable` and `!which python` both point to the same location: `/local_disk0/.ephemeral_nfs/envs/pythonEnv-<random string>/bin/python. But `!jupyter kernlespec list` returns: 

Available kernels: python3 /databricks/python3/share/jupyter/kernels/python3

However, running the command:

!source activate pip install ipykernel python -m ipykernel install --user --name mypkg --display-name "mypkg"

 does not make my local project available as a package.

Is there any other command I can try? Especially any solution that makes my current project available more broadly across all executable programs within my project? I'd prefer not to have to adjust on a script-by-script or notebook-by-notebook basis (which is also a downside of the `sys.path.append` solution), but this is effectively what I'm doing rihgt now due to a lack of alternatives.

Thank you