cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

How to solve Py4JJavaError: An error occurred while calling o5082.csv. : org.apache.spark.SparkException: Job aborted. when writing to csv

laus
New Contributor III

Screenshot 2022-03-31 at 17.33.26Hi ,

I get the error: Py4JJavaError: An error occurred while calling o5082.csv.

: org.apache.spark.SparkException: Job aborted. when writing to csv.

Screenshot below with detail error.

Any idea how to solve it?

Thanks!

1 ACCEPTED SOLUTION

Accepted Solutions

Noopur_Nigam
Valued Contributor II
Valued Contributor II

Please try output.coalesce(1).write.option("header","true").format("csv").save("path")

It seems to be same to https://community.databricks.com/s/topic/0TO3f000000CjVqGAK/py4jjavaerror

View solution in original post

4 REPLIES 4

Kaniz
Community Manager
Community Manager

Hi @Laura Blancarte​, Looks like you want to save your dataframe as csv. Did you try to download the preview?

rahrthr12
New Contributor II

i am also facing same issue

User16764241763
Honored Contributor

Hello @Laura Blancarte​ , @Rahul Rathore​ 

Would you mind sharing sample data from the input dataframe that is producing this error while saving?

Noopur_Nigam
Valued Contributor II
Valued Contributor II

Please try output.coalesce(1).write.option("header","true").format("csv").save("path")

It seems to be same to https://community.databricks.com/s/topic/0TO3f000000CjVqGAK/py4jjavaerror