I have a job in Databricks which completed successfully but the data is not been written into the target table, I have checked all the possible ways, each n every thing is correct in the code, target table name, source table name, etc etc. It is a Full load, I can see only full load is getting failed, delta load is loading the data, only june 26th it has loaded the full load data, on next day it failed as " org.apache.spark.SparkException: [SPARK_JOB_CANCELLED] Job 13 cancelled because Task 136 in stage 23 exceeded the maximum allowed ratio of input to output records (1 to 0, max allowed 1 to -1); this limit can be modified with configuration parameter spark.databricks.queryWatchdog.outputRatioThreshold" it failed for two three runs, after that it got successfully but from then the data is not written into the target table, job is completing successfully but no data in table. Can anyone pls help me.
