- 6766 Views
- 16 replies
- 5 kudos
toPandas() causes IndexOutOfBoundsException in Apache Arrow
Using DBR 10.0When calling toPandas() the worker fails with IndexOutOfBoundsException. It seems like ArrowWriter.sizeInBytes (which looks like a proprietary method since I can't find it in OSS) calls arrow's getBufferSizeFor which fails with this err...
- 6766 Views
- 16 replies
- 5 kudos
Latest Reply
- 5 kudos
I am also facing the same issue, I have applied the config: `spark.sql.execution.arrow.pyspark.enabled` set to `false`, but still facing the same issue. Any Idea, what's going on???. Please help me out....org.apache.spark.SparkException: Job aborted ...
- 5 kudos