Dlt
New Contributor III

Hello , 

If i refer to above code you created then error is like below pyspark.errors.exceptions.AnalysisException: Failed to read dataset 'Temp_Table'. Dataset is not defined in the pipeline.  for each of 5 Temp tables 

Below is flow at high level for my DLT Pipeline. 

Step1 - 5 Bronze level tables are created and loaded from JSON files

Step2 - 5 Temp tables are created from 5 bronze tables ( created in step 1)  above with Boolean bad flag ( derived)  

Step3 -  5 Clean and 5 Quarantine tables are created by seperating Good & Bad data based on Bad Flag.

Step4 - 5 Gold layer tables are created from 5 clean tables created in Step 3.

Earlier i had separate notebook for each step which worked great. But when i combined all these into one notebook i am running into issues which i am NOT able to understand.  Each table is in separate cell in all steps as such.