Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-22-2026 07:51 AM
Just to share knowledge with other readers, I think the best option for now(until Databricks supports adding project root to PYTHONPATH) is to refactor the project to have custom modules in a separate folder, create a wheel when building DAB and declare as dependency of pipeline/job config.
I followed the instruction in this page, but struggled to make it work. It turned out dependency config is different if pipeline runs on serverless. Instead of using "libraries" as in the instruction , I have to use "environment".
resources:
pipelines:
<pipeline name>
environment:
dependencies:
- <path to wheel file>I wish the instruction included this example for serverless, or Databricks supports same configuration regardless of compute type.