Hi @himanmon ,

This is caused because of S3 limit on segment count. The part files can be numbered only from 1 to 10000

After Setting spark.hadoop.fs.s3a.multipart.size to 104857600. , did you RESTART the cluster? Because it'll only work when the cluster is restarted. 

Also, before sending this file you can try to compress it using gzip