Databricks Community

Maxrb

Hi,We are using Databricks asset bundles, and we create functions which we import in notebooks, for instance:from utils import helperswhere utils is just a folder in our root. When running this with source: WORKSPACE, it will fail to resolve the impo...

Maxrb

Hi,After moving from Databricks runtime 17.1 to 17.2 suddenly my pkgutils walk_packages doesn't identify any packages within my repository anymore.This is my example code:import pkgutil import os packages = pkgutil.walk_packages([os.getcwd()]) print...

Maxrb

Hey @Louis_Frolio,Thanks for thinking along, the whole idea is that this package is not installed as a jar, wheel or something else, but it's a living module in my repository. For production I don't think I will have this issue as I install my repo ...

Maxrb

I did a bit of deep dive into the source code of the pkgutils walk_package, and I noticed this happening:def get_importer(path_item): path_item = os.fsdecode(path_item) try: importer = sys.path_importer_cache[path_item] except: ...

Maxrb

Hi @Louis_Frolio ,Unfortunately Whatever I am doing, add all paths etc, trying all your solutions it just simply doesn't work. When I run pkgutil on for instance a pyspark.sql packages __path__ it simply works. For me it looks like anything inside th...

Maxrb

Thanks for the detailed answer @Louis_Frolio ,Unfortunately, none of this is working, I have a notebook in my repo root, I checked all the sys.path, the cwd and, did all the options you mentioned and still it doesn't work in dbr 17.2+. Simply put I s...

Databricks Community

User Stats

User Activity

Import functions in databricks asset bundles using source: WORKSPACE

pkgutils walk_packages stopped working in DBR 17.2

Re: pkgutils walk_packages stopped working in DBR 17.2

Re: pkgutils walk_packages stopped working in DBR 17.2

Re: pkgutils walk_packages stopped working in DBR 17.2

Re: pkgutils walk_packages stopped working in DBR 17.2