Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-14-2021 01:59 AM
Hello guys, I'm trying to read JSON files from the s3 bucket. but no matter what I try I get Query returned no result or if I don't specify the schema I get unable to infer a schema.
I tried to mount the s3 bucket, still not works.
here is some code that I tried:
df = spark.read.json('dbfs:/mnt/path_to_json', multiLine="true", schema= json_schema)
df = spark.read.option('multiline','true').format('json').load(path_to_json)
df = spark.read.json('s3a:// path_to _json, multiline=True)
display(df)The json file look like this:
{
'key1' : 'value1',
'key2' : 'value2',
...
}
hope you guys can help me,
Thanks!
**EDIT**: inside the JSON i have string value that contains " \ " which throw corrupted error, is there any way to overcome this without change the value for the specific key?