Noisy error in Databricks.

[alvaro_databric]

Hello all, 

I need some help figuring out the exact cause of this error. While in the middle of a complex scala transformation notebook in Azure Databricks, from time to time i get an error which looks like this, can someone help?

Job aborted due to stage failure: $linee899685e104341378f258eaf5a3e740579/$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.class

23/09/15 10:15:32 ERROR Uncaught throwable from user code: org.apache.spark.SparkException: Job aborted due to stage failure: Task 59 in stage 141.0 failed 4 times, most recent failure: Lost task 59.3 in stage 141.0 (TID 14396) (10.8.24.23 executor 4): org.apache.spark.repl.RemoteClassLoaderError: $linee899685e104341378f258eaf5a3e740579/$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.class

at org.apache.spark.repl.ExecutorClassLoader$$anon$1.toClassNotFound(ExecutorClassLoader.scala:156)

at org.apache.spark.repl.ExecutorClassLoader$$anon$1.read(ExecutorClassLoader.scala:143)

at java.io.FilterInputStream.read(FilterInputStream.java:107)

at org.apache.spark.repl.ExecutorClassLoader.readAndTransformClass(ExecutorClassLoader.scala:263)

at org.apache.spark.repl.ExecutorClassLoader.findClassLocally(ExecutorClassLoader.scala:217)

at org.apache.spark.repl.ExecutorClassLoader.findClass(ExecutorClassLoader.scala:115)

at java.lang.ClassLoader.loadClass(ClassLoader.java:419)

at java.lang.ClassLoader.loadClass(ClassLoader.java:352)

at java.lang.Class.getDeclaredFields0(Native Method)

at java.lang.Class.privateGetDeclaredFields(Class.java:2583)

at java.lang.Class.getDeclaredField(Class.java:2068)

at java.io.ObjectStreamClass.getDeclaredSUID(ObjectStreamClass.java:1871)

at java.io.ObjectStreamClass.access$700(ObjectStreamClass.java:79)

at java.io.ObjectStreamClass$3.run(ObjectStreamClass.java:506)

at java.io.ObjectStreamClass$3.run(ObjectStreamClass.java:494)

at java.security.AccessController.doPrivileged(Native Method)

... 209 more

[Kaniz]

Hi @alvaro_databric ,  Error message suggests error related to execution context in Azure Databricks.

• Steps to troubleshoot error:
 - Check cluster configuration (nodes, memory, CPU cores)
 - Review job logs for specific error messages or exceptions
 - Review code and transformations for potential issues or errors
 - Verify data and input/output paths
 - Check for resource contention if multiple jobs running concurrently
• If above steps don't help, further investigation or reaching out to support by filing a support ticket may be necessary.

[alvaro_databric]

Hi,

Cluster configuration, user code and input data has remain unchanged. I am indeed running concurrent jobs using the same source table and generate an output table for every individual job. Can you elaborate on what you mean with resource contention? I do not understand how this applies to Databricks jobs.

Thanks in advance!