Use one Auto Loader streaming table per object type, generated from a governed configuration registry; not dbutils.fs.ls. Each object type needs an independent schema and checkpoint. Auto Loader cannot use one CSV stream to infer unrelated schemas a...
That version should work although documentation doesn't list it yet. Can you try out? If it fails I would recommend opening a support case with your workspace ID and the exact error message so the team can verify the allowlist status.
Hi @ADB0513 With %run, you dont need to pass anything. %run runs the child inline in the same Python namespace, so the catalog variable set in the main notebook is already visible in the child. Just use it.
# main
catalog = "dev_catalog"
# %run ./chi...
Hi @tomodachilifero ,The TLDR is: Land CSVs in governed Unity Catalog storage, use a Lakeflow pipeline or Auto Loader for repeatable ingestion, analyze with SQL/PySpark notebooks, and publish an AI/BI dashboard on a SQL warehouse.CSV or exported Anal...
