Databricks Community

david_aspegren · 09-02-2026

When a pipeline contains more than one create_auto_cdc_from_snapshot_flow flow (each driven by a custom Python next_snapshot_and_version function), flow resolution fails intermittently/consistently with: RuntimeError: The original Spark session is be...

david_aspegren · 09-17-2026

Hi again, it seems to have regressed on me, my pipeline is failing again, anybody else seeing this?

david_aspegren · 09-02-2026

thanks guys, what i have now is:_snapshot_versions = sorted( row[0] for row in spark.read.table(SOURCE_TABLE) .select(VERSION_COLUMN) .distinct() .collect())_snapshots = { version: _cl...

david_aspegren · 09-02-2026

Thank you! I am still doing something wrong though, my function is now called like so:dp.create_auto_cdc_from_snapshot_flow( target=SILVER_TABLE, source=lambda latest_version: next_locations_snapshot_and_version(spark, latest_version), keys=...

Databricks Community

User Stats

User Activity

create_auto_cdc_from_snapshot_flow Python session resolution fails if having multiple snapshot flows

Re: create_auto_cdc_from_snapshot_flow Python session resolution fails if having multiple snapshot f

Re: create_auto_cdc_from_snapshot_flow Python session resolution fails if having multiple snapshot f

Re: create_auto_cdc_from_snapshot_flow Python session resolution fails if having multiple snapshot f