Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-05-2026 09:51 AM
@Hubert-Dudek, thank you for your response. I really appreciate it.
However, I still cannot get the import to work even after I follow your instructions. Here is the folder structure:
The transformer code is below:
from pyspark import pipelines as dp
from project.utils.logger import get_logger
#from utils.logger import get_logger
@dp.table
def sample_users():
logger = get_logger(__name__)
logger.info('Running sample_users transformer')
return (
spark.read.table("samples.wanderbricks.users")
.select("user_id", "email", "name", "user_type")
)After deploying the bundle, I tried running the job but it still fails to import the logger module.
Traceback (most recent call last):
File "/Workspace/Users/.../.bundle/import_test/dev/files/src/project/pipelines/import_test/transformations/sample_users.py", cell 1, line 3
1 from pyspark import pipelines as dp
----> 3 from project.utils.logger import get_logger
4 #from utils.logger import get_logger
6 @dp.table
7 def sample_users():
File "/databricks/python_shell/lib/dbruntime/autoreload/discoverability/hook.py", line 71, in AutoreloadDiscoverabilityHook._patched_import(self, name, *args, **kwargs)
65 if not self._should_hint and (
66 (module := sys.modules.get(absolute_name)) is not None and
67 (fname := get_allowed_file_name_or_none(module)) is not None and
68 (mtime := os.stat(fname).st_mtime) > self.last_mtime_by_modname.get(
69 absolute_name, float("inf")) and not self._should_hint):
70 self._should_hint = True
---> 71 module = self._original_builtins_import(name, *args, **kwargs)
72 if (fname := fname or get_allowed_file_name_or_none(module)) is not None:
73 mtime = mtime or os.stat(fname).st_mtime
ModuleNotFoundError: No module named 'project'
Error: [update_progress] Update 803be7 is FAILED.I tried importing from both "project.utils.logger" and "utils.logger" and neither worked.
Can you help me figure out what I'm doing wrong?