I have data on s3 and i'm using autoloader to load the data. My json docs have fields which are array of structures.

When I don't specify any schema the whole data is stored as strings even the array of structures are just a blob of string making it difficult to process with pyspark dataframe.

When i do specify a schema to autoload, the whole table is null.

Did anyone face any similar issues?