cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Importing python to DLT - Not working with DLT Pipeline

AxelBrsn
New Contributor III

Hello, we are trying to adapt our developments (notebook with delta tables), into Delta Live Tables Pipelines.

We tried to import Python files that are very useful for data transformations (silver data cleaning, for example) :

  • From the Cluster (run manually the declarative notebook of DLT) : It's working.
  • From the DLT Pipeline : It's not, it can't find and reach the python files in question.

We tried to import it as Python files, and also as Python library (wheel file extension) :

  • Images are available in attachment.

Thank you for your time, and your help!

1 ACCEPTED SOLUTION

Accepted Solutions

AxelBrsn
New Contributor III

The solution is to import from Python but also add the python file in the Pipeline settings, in the list of source code.

View solution in original post

2 REPLIES 2

AxelBrsn
New Contributor III

Hello @Retired_mod, thanks for the answer, but it's not completely clear on my side.
I want to import, not to install ?

Why the import is not accessible from the DLT Pipeline?

AxelBrsn
New Contributor III

The solution is to import from Python but also add the python file in the Pipeline settings, in the list of source code.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group