Hello all,
I need some help figuring out the exact cause of this error. While in the middle of a complex scala transformation notebook in Azure Databricks, from time to time i get an error which looks like this, can someone help?
Job aborted due to stage failure: $linee899685e104341378f258eaf5a3e740579/$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.class
23/09/15 10:15:32 ERROR Uncaught throwable from user code: org.apache.spark.SparkException: Job aborted due to stage failure: Task 59 in stage 141.0 failed 4 times, most recent failure: Lost task 59.3 in stage 141.0 (TID 14396) (10.8.24.23 executor 4): org.apache.spark.repl.RemoteClassLoaderError: $linee899685e104341378f258eaf5a3e740579/$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.class
at org.apache.spark.repl.ExecutorClassLoader$$anon$1.toClassNotFound(ExecutorClassLoader.scala:156)
at org.apache.spark.repl.ExecutorClassLoader$$anon$1.read(ExecutorClassLoader.scala:143)
at java.io.FilterInputStream.read(FilterInputStream.java:107)
at org.apache.spark.repl.ExecutorClassLoader.readAndTransformClass(ExecutorClassLoader.scala:263)
at org.apache.spark.repl.ExecutorClassLoader.findClassLocally(ExecutorClassLoader.scala:217)
at org.apache.spark.repl.ExecutorClassLoader.findClass(ExecutorClassLoader.scala:115)
at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
at java.lang.Class.getDeclaredFields0(Native Method)
at java.lang.Class.privateGetDeclaredFields(Class.java:2583)
at java.lang.Class.getDeclaredField(Class.java:2068)
at java.io.ObjectStreamClass.getDeclaredSUID(ObjectStreamClass.java:1871)
at java.io.ObjectStreamClass.access$700(ObjectStreamClass.java:79)
at java.io.ObjectStreamClass$3.run(ObjectStreamClass.java:506)
at java.io.ObjectStreamClass$3.run(ObjectStreamClass.java:494)
at java.security.AccessController.doPrivileged(Native Method)
... 209 more
