Just to share knowledge with other readers, I think the best option for now(until Databricks supports adding project root to PYTHONPATH) is to refactor the project to have custom modules in a separate folder, create a wheel when building DAB and decl...
@rcdatabricks,I tried that.But import still fails.ile "/Workspace/Users/.../.bundle/import_test/dev/files/src/project/pipelines/import_test/transformations/sample_users.py", cell 1, line 3
1 from pyspark import pipelines as dp
----> 3 from proj...
@Hubert-Dudek, thank you for your response. I really appreciate it.However, I still cannot get the import to work even after I follow your instructions. Here is the folder structure:The transformer code is below:from pyspark import pipelines as dp
f...
I have a related question.I'm new to Databricks platform. I struggle with PYTHONPATH issue as the original poster raised. I understand using sys.path.append(...) is one approach for notebook. This is acceptable for ad-hoc interactive session, but thi...
