I did have the same exact error but I don't have any parallelism especially in that notebook. The failure was in the subsequent notebook for some other parallel runs. like below1 --> 2,3,4,5 --> 6 --> 7 . This is how my pipeline looks like and 2,3,4,...