Cannot import relative python paths

[georgef]

Hello,

Some variations of this question have been asked before but there doesn't seem to be an answer for the following simple use case:

I have the following file structure on a Databricks Asset Bundles project:

src
--dir1
----file1.py
--dir2
----file2.py

There doesn't seem to be a way to import anything from file1.py to file2.py. I have tried (inside file2.py):

from src.dir1.file1 import something
from dir1.file1 import something
from ..dir1.file1 import something

None of these seem to work. I have tried appending os.abspath("..") and "../.." to sys.path, as well as passing ${workspace.file_path} to the task and appending that, to no avail.

Note that relative imports on the same level work so the following both work:

# src
# --file1.py
# --file2.py
# We are on file1.py
from file2 import something

# src
# --file1.py
# --dir2
# ----file2.py
# We are on file1.py
from dir2.file2 import something

Is something weird going on with the import paths when deploying DABs? I'd rather avoid having to deploy a full-on wheel if I can avoid it, as it adds a lot of unnecessary boilerplate.

Thank you!

[georgef · Accepted solution]

Unfortunately none of the methods proposed here worked, so we ended up having to wrap our code in packages and installing those in the cluster before running. Not an ideal workflow, but works for projects that are easily decoupled from their orchestration.

[m997al]

Hi.  This was a long-standing issue for me too.  This solution may not be what is desired, but it works perfectly for my needs.

In my python code, I have this structure:

if __name__ == '__main__':

    # directory structure where "mycode" is this code here, trying to import "mymodule"
    #  > main
    #    -- mymodule
    #    > subdir
    #      -- mycode

    # Get the current directory
    current_directory = os.getcwd()

    # Get the parent directory
    parent_directory = os.path.abspath(os.path.join(current_directory, '..'))

    # Add the parent directory to sys.path
    sys.path.append(parent_directory)

    from mymodule import myfunction