Please look this StackOverflow answer for the most effective way to use the HDFS FileUtils.copyMerge() command:
http://stackoverflow.com/a/41785085/501113
Please look this StackOverflow answer for the most effective way to use the HDFS FileUtils.copyMerge() command:
http://stackoverflow.com/a/41785085/501113
THIS IS TERRIBLE ADVICE. DO NOT USE the DataFrame methods of .coalesce(1) or .repartition(1) except for very small data sets. Instead, use the hdfs merge mechanism via FileUtils.copyMerge(). This solution on StackOverflow correctly identifies how to...
The way to turn off the default escaping of the double quote character (") with the backslash character (\) - i.e. to avoid escaping for all characters entirely, you must add an .option() method call with just the right parameters after the .write() ...
Your answer is actually not only incorrect, it causes the JSON content to become corrupt. So, while it might have solved a highly specific problem you had at the time you were doing this, it isn't a general solution. I have come up with a general so...
