- 3160 Views
- 5 replies
- 4 kudos
Resolved! pyspark streaming failed for now reason
Hi everyone, I have a pyspark streaming reading from an aws kinesis that suddenly failed for no reason (I mean, we did not make any changes in the last time).It is giving the following error: ERROR MicroBatchExecution: Query kinesis_events_prod_bronz...
- 3160 Views
- 5 replies
- 4 kudos
Latest Reply
- 4 kudos
@patricio tojo​ I've the same problem, however in my case is after migrating into unity catalog. Need to investigate a little more but adding this to my spark job, it works:spark.conf.set("spark.databricks.delta.state.corruptionIsFatal", False)
- 4 kudos