DLT Pipeline Stopped working

maninegi05
New Contributor II

Hello, 

Suddenly our DLT pipelines we're getting failures saying that

LookupError: Traceback (most recent call last):

    result_df = result_df.withColumn("input_file_path", col("_metadata.file_path")).withColumn(
                                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^

LookupError: <ContextVar name='parent_header'

For the failing pipelines, when looking at the Update Details - > Logs -> Configuration tab, that the failed pipelines take runtime "dlt:16.4.10-delta-pipelines-dlt-release-dp-20251009-rc0-commit-8c6b818-image-4a72116". 

Did something change on the Databricks end? For us nothing changed in the settings and seems like a sudden disruption of DLT pipelines that were previously just running successfully.

Thank you in advance.