Hi,

I have a problem with writing an excel file into the mounted file.

after 10 mins I see the Driver is up but is not responsive, likely due to GC on the log events.

I'm using the following script:

df.repartition(1).write

 .format("com.crealytics.spark.excel")

 .mode("overwrite") // Optional, default: overwrite.

 .option("header", "true")

 .option("maxRowsInMemory", 1000)

 .save("/mnt/IngestExelFiles/output_fulldf.xlsx")

the feature of the cluster is:

8.4 (includes Apache Spark 3.1.2, Scala 2.12)

Driver type:

85 Gb memory/ 8 cores

the approximate size of the excel file is 200 MB.

would you help to solve this problem?

Thanks