How to solve Py4JJavaError: An error occurred while calling o5082.csv. : org.apache.spark.SparkException: Job aborted. when writing to csv

laus
New Contributor III

‎03-31-2022 09:38 AM

Screenshot 2022-03-31 at 17.33.26Hi ,

I get the error: Py4JJavaError: An error occurred while calling o5082.csv.

: org.apache.spark.SparkException: Job aborted. when writing to csv.

Screenshot below with detail error.

Any idea how to solve it?

Thanks!

Labels:
Preview file
977 KB
0 Kudos

rahrthr12
New Contributor II

‎04-07-2022 11:06 AM

i am also facing same issue

0 Kudos

User16764241763
Databricks Employee
Databricks Employee

‎04-26-2022 05:41 AM

Hello @Laura Blancarte​ , @Rahul Rathore​ 

Would you mind sharing sample data from the input dataframe that is producing this error while saving?

Noopur_Nigam
Databricks Employee
Databricks Employee

‎05-13-2022 07:52 AM

Please try output.coalesce(1).write.option("header","true").format("csv").save("path")

It seems to be same to https://community.databricks.com/s/topic/0TO3f000000CjVqGAK/py4jjavaerror

View solution in original post

TechnoCoder
New Contributor II
In response to Noopur_Nigam

‎07-01-2025 08:01 AM

Traceback (most recent call last):

File "C:\Users\Administrator\Documents\practice code\spark_pract\read_merge_print copy.py", line 49, in <module>
merged_df.write.mode("overwrite").option("header", True).csv("output")

py4j.protocol.Py4JJavaError: ... ExitCodeException exitCode=-1073741515

 

 

I tried your code still getting same error,

Also checked, Hadoop.dll,winutils.exe,spark home variable at environmental variable,

 

 

All exist.....still getting error,

 

I have spark lattest version,

Hadoop 3.3.0

Jdk 17

 

 

Please help me if you can 

0 Kudos