So I've been having some issues reading a json file that's been provided to the business with another nesting layer, so instead of a json being an:
- 'array of objects' -> [ {} ,{} ,{} ]
- It's an 'array of arrays of objects' -> [ [ {}, {} ,{} ], [ {} ,{} ,{} ] ]
While the first is alright to read with the multiline option with spark, the second case simply comes with the correct column schema, thought every columns is just a null value (actual file content looks good)
I've so far tried to create a custom struct schema to deal with the extra layer, but not had any luck to get it to work. Just returns nulls.
Is there something obvious that i'm missing?
