Databricks Community

Maxrb · 06-22-2026

Hi,I am using autoloader to load parquet files into my unity catalog with the following settings:.option("cloudFiles.format", "parquet") .option("cloudFiles.inferColumnTypes", "true") .option("cloudFiles.schemaEvolutionMode", "addNewColumnsWithTypeWi...

Maxrb · 01-06-2026

Hi,We are using Databricks asset bundles, and we create functions which we import in notebooks, for instance:from utils import helperswhere utils is just a folder in our root. When running this with source: WORKSPACE, it will fail to resolve the impo...

Maxrb · 12-18-2025

Hi,After moving from Databricks runtime 17.1 to 17.2 suddenly my pkgutils walk_packages doesn't identify any packages within my repository anymore.This is my example code:import pkgutil import os packages = pkgutil.walk_packages([os.getcwd()]) print...

Maxrb · 06-22-2026

@balajij8 Thanks for your replay.I do see what you mean, at the same time I see that _rescued_data works for some type mismatches, which is why I am confused. Do you have any idea why it works when I get string data in a integer column but not for th...

Maxrb · 12-19-2025

Hey @Louis_Frolio,Thanks for thinking along, the whole idea is that this package is not installed as a jar, wheel or something else, but it's a living module in my repository. For production I don't think I will have this issue as I install my repo ...

Maxrb · 12-19-2025

I did a bit of deep dive into the source code of the pkgutils walk_package, and I noticed this happening:def get_importer(path_item): path_item = os.fsdecode(path_item) try: importer = sys.path_importer_cache[path_item] except: ...

Maxrb · 12-19-2025

Hi @Louis_Frolio ,Unfortunately Whatever I am doing, add all paths etc, trying all your solutions it just simply doesn't work. When I run pkgutil on for instance a pyspark.sql packages __path__ it simply works. For me it looks like anything inside th...

Maxrb · 12-18-2025

Thanks for the detailed answer @Louis_Frolio ,Unfortunately, none of this is working, I have a notebook in my repo root, I checked all the sys.path, the cwd and, did all the options you mentioned and still it doesn't work in dbr 17.2+. Simply put I s...

Databricks Community

User Stats

User Activity

Autoloader [FAILED_READ_FILE.PARQUET_COLUMN_DATA_TYPE_MISMATCH]

Import functions in databricks asset bundles using source: WORKSPACE

pkgutils walk_packages stopped working in DBR 17.2

Re: Autoloader [FAILED_READ_FILE.PARQUET_COLUMN_DATA_TYPE_MISMATCH]

Re: pkgutils walk_packages stopped working in DBR 17.2

Re: pkgutils walk_packages stopped working in DBR 17.2

Re: pkgutils walk_packages stopped working in DBR 17.2

Re: pkgutils walk_packages stopped working in DBR 17.2