topic Re: Importing Python files into another Workspace Python file does not work in Warehousing & Analytics

Importing Python files into another Workspace Python file does not work

sachamourier — Fri, 20 Sep 2024 14:27:58 GMT

I have created Python modules containing some Python functions and I would like to import them from a notebook contained in the Workspace.

For example, I have a "etl" directory, containing a "snapshot.py" file with some Python functions, and an empty "__init__.py" file as well. When working inside my repo in Databricks, the "from etl.snapshot import *" Python command works with no issue and I am able to call my functions. However, when running that same command from the Workspace, I get a "Module not found" error. I have tried to use the "sys.path.append()" approach as suggested in many forums or in multiple YouTube videos but this still does not work.

I am looking for a solution, as I would like to avoid using the %run command.

Thanks a lot in advance for your help,
Sacha

Re: Importing Python files into another Workspace Python file does not work

filipniziol — Fri, 20 Sep 2024 14:45:45 GMT

Hi @sachamourier ,
It will work, but you need carefully craft path to sys.path.append(), you even do not need __init__.py to make it work.
Try to hard-code the path to the snapshot.py in workspace.

Add this to your notebook:

import sys
import os

absolute_directory_path = os.path.normpath("/Workspace/<path to directory where the snapshot.py file is located>")

if absolute_directory_path not in sys.path:
    sys.path.append(absolute_directory_path)

And then you make your import:

import snapshot

Re: Importing Python files into another Workspace Python file does not work

sachamourier — Fri, 20 Sep 2024 14:58:45 GMT

Hi @filipniziol ,
Thank you for your response. I have tried but this still does not work. The path gets added to "sys.path" though. You can find attached images of the issue.
Sacha

Re: Importing Python files into another Workspace Python file does not work

filipniziol — Fri, 20 Sep 2024 15:20:56 GMT

Hi @sachamourier,
Your snapshot.py is Databricks Notebook, and not a .py file.

Check the icons.

Try adding snapshot.py as a file. It will work:

Also, after you replace snapshot notebook with snapshot file, make sure to clear state in the notebook that you use to reference the snapshot.py: