cancel
Showing results for 
Search instead for 
Did you mean: 
Warehousing & Analytics
Engage in discussions on data warehousing, analytics, and BI solutions within the Databricks Community. Share insights, tips, and best practices for leveraging data for informed decision-making.
cancel
Showing results for 
Search instead for 
Did you mean: 

Importing Python files into another Workspace Python file does not work

sachamourier
New Contributor II

I have created Python modules containing some Python functions and I would like to import them from a notebook contained in the Workspace. 

For example, I have a "etl" directory, containing a "snapshot.py" file with some Python functions, and an empty "__init__.py" file as well. When working inside my repo in Databricks, the "from etl.snapshot import *" Python command works with no issue and I am able to call my functions. However, when running that same command from the Workspace, I get a "Module not found" error. I have tried to use the "sys.path.append()" approach as suggested in many forums or in multiple YouTube videos but this still does not work.

I am looking for a solution, as I would like to avoid using the %run command. 

Thanks a lot in advance for your help,
Sacha

3 REPLIES 3

filipniziol
Contributor

Hi @sachamourier ,
It will work, but you need carefully craft path to sys.path.append(), you even do not need __init__.py to make it work.
Try to hard-code the path to the snapshot.py in workspace.

Add this to your notebook: 

import sys
import os

absolute_directory_path = os.path.normpath("/Workspace/<path to directory where the snapshot.py file is located>")

if absolute_directory_path not in sys.path:
    sys.path.append(absolute_directory_path)

 And then you make your import:

 

import snapshot

 

 

 

Hi @filipniziol ,
Thank you for your response. I have tried but this still does not work. The path gets added to "sys.path" though. You can find attached images of the issue.
Sacha

Screenshot 2024-09-20 165613.pngScreenshot 2024-09-20 165645.png

Hi @sachamourier,
Your snapshot.py is Databricks Notebook, and not a .py file.

Check the icons.

filipniziol_0-1726845391181.png

Try adding snapshot.py as a file. It will work:

filipniziol_1-1726845462675.png

Also, after you replace snapshot notebook with snapshot file, make sure to clear state in the notebook that you use to reference the snapshot.py:

filipniziol_0-1726845646839.png

 



Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group