DLT - Importing Python Package

YS1
Contributor

Hello,

I'm creating a DLT pipeline where I read a Kafka stream, perform transformations using UDFs, and save the data in multiple tables. When I define the functions directly in the same notebook, the code works fine. However, if I move the code into a Python file and try to import it as a package, the pipeline fails with the following error:

YS1_1-1723071739598.png

I've tried renaming the files and changing their locations -although the pipeline and the package are in the same directory-

Despite these efforts, nothing worked. Here is how I'm trying to import the module:

YS1_0-1723071683421.png

Any insights or solutions would be greatly appreciated!

szymon_dybczak
Esteemed Contributor III

Hi @YS1 ,

Have you added the python file in the Pipeline settings, in the list of source code? 

Slash_1-1723118449935.png

 

 

 

 

yes, I did, the pipeline would work for seconds and then fails with the same error "No module named 'eventlog_utils'". And that's why it's confusing because I tried this on another pipeline with different setup/functions and it works fine!