Getting NO_TABLES_IN_PIPELINE error in Lakeflow Declarative Pipelines

cpollock
New Contributor III

Yesterday (10/1) starting around 12 PM EST we starting getting the following error in our Lakeflow Declarative Pipelines (LDP) process.  We get this in environments where none of our code has changed.  I found some info on the serverless compute about the Databricks Runtime version before the error and after.

This is the version on our last successful run:  dlt:16.4.9-delta-pipelines-dlt-release-dp-20250918-rc0-commit-015f1e5-image-20d2b81

And this is the version on all new runs getting the error:  dlt:16.4.9-delta-pipelines-dlt-release-dp-20250925-rc0-commit-a18ed15-image-7ea1ff2

 

Also, the LDPs are based off of a python meta-data driven process (similar to DLT-META) that dynamically creates tables (in bronze and silver) using the dlt.table and dlt.create_streaming_table functions.  Have tried many different things but it seems debugging LDPs is very challenging since you can't do print and logging statements (at least not that I've seen).