Importing Python files into another Workspace Python file does not work

[sachamourier]

I have created Python modules containing some Python functions and I would like to import them from a notebook contained in the Workspace. 

For example, I have a "etl" directory, containing a "snapshot.py" file with some Python functions, and an empty "__init__.py" file as well. When working inside my repo in Databricks, the "from etl.snapshot import *" Python command works with no issue and I am able to call my functions. However, when running that same command from the Workspace, I get a "Module not found" error. I have tried to use the "sys.path.append()" approach as suggested in many forums or in multiple YouTube videos but this still does not work.

I am looking for a solution, as I would like to avoid using the %run command. 

Thanks a lot in advance for your help,
Sacha

[filipniziol]

Hi @sachamourier ,
It will work, but you need carefully craft path to sys.path.append(), you even do not need __init__.py to make it work.
Try to hard-code the path to the snapshot.py in workspace.

Add this to your notebook: 

import sys
import os

absolute_directory_path = os.path.normpath("/Workspace/<path to directory where the snapshot.py file is located>")

if absolute_directory_path not in sys.path:
    sys.path.append(absolute_directory_path)

 And then you make your import:

import snapshot

[sachamourier]

Hi @filipniziol ,
Thank you for your response. I have tried but this still does not work. The path gets added to "sys.path" though. You can find attached images of the issue.
Sacha

[filipniziol]

Hi @sachamourier,
Your snapshot.py is Databricks Notebook, and not a .py file.

Check the icons.

Try adding snapshot.py as a file. It will work:

Also, after you replace snapshot notebook with snapshot file, make sure to clear state in the notebook that you use to reference the snapshot.py:

[rdokala]

You are absolutely right! that was not .py file.

[dee_vy]

Hello,

I was facing the same issues as the OP and was unable to resolve them using the suggested solutions.

In my case, the root cause turned out to be the cluster configuration rather than pytest or Databricks imports.

I verified this by switching to a different cluster. The new cluster was able to see the project files correctly, imports from the src folder worked, and pytest ran successfully without any changes to the project structure.

The original cluster could see the project directories but was not resolving the file contents correctly, which resulted in ModuleNotFoundError and related import issues.

Sharing this in case it helps someone facing a similar problem. Thank you!