How to solve Py4JJavaError: An error occurred while calling o5082.csv. : org.apache.spark.SparkException: Job aborted. when writing to csv

[laus]

Hi ,

I get the error: Py4JJavaError: An error occurred while calling o5082.csv.

: org.apache.spark.SparkException: Job aborted. when writing to csv.

Screenshot below with detail error.

Any idea how to solve it?

Thanks!

[rahrthr12]

i am also facing same issue

[User16764241763]

Hello @Laura Blancarte​ , @Rahul Rathore​ 

Would you mind sharing sample data from the input dataframe that is producing this error while saving?

[Noopur_Nigam · Accepted solution]

Please try output.coalesce(1).write.option("header","true").format("csv").save("path")

It seems to be same to https://community.databricks.com/s/topic/0TO3f000000CjVqGAK/py4jjavaerror

[TechnoCoder]

Traceback (most recent call last):

File "C:\Users\Administrator\Documents\practice code\spark_pract\read_merge_print copy.py", line 49, in <module>
merged_df.write.mode("overwrite").option("header", True).csv("output")

py4j.protocol.Py4JJavaError: ... ExitCodeException exitCode=-1073741515

I tried your code still getting same error,

Also checked, Hadoop.dll,winutils.exe,spark home variable at environmental variable,

All exist.....still getting error,

I have spark lattest version,

Hadoop 3.3.0

Jdk 17

Please help me if you can