@Kamal Kumar​ :

The error message suggests that the JSON document size is exceeding the maximum allowed size of 400MB. This could be caused by one or more documents in your JSON file being larger than this limit. It is not a bug, but a limitation set by the Photon JSON reader.

To resolve the issue, you could try splitting the JSON file into smaller chunks and processing them separately. You can do this using tools like jq or split in Linux or Python libraries like ijson. Alternatively, you could consider using a different JSON reader that does not have this limitation.

Another possible cause of this error could be memory issues. If your system does not have enough memory to process the large JSON file, it may cause the error you are seeing. In this case, you may need to increase the memory allocated to your Spark job or optimize your code to use less memory.