https://community.databricks.com/t5/get-started-discussions/how-to-handle-complex-json-schema/m-p/57635#M2252 <P>I have a mounted external directory that is an s3 bucket with multiple subdirectories containing call log files in json format.&nbsp; The files are irregular and complex, when i try to use spark.read.json or spark.sql (SELECT *) i get the&nbsp;UNABLE_TO_INFER_SCHEMA error.&nbsp; the files are too complex to try and build a schema manually, plus there are thousands of files.&nbsp; what is the best approach for creating a dataframe with this data?</P> Thu, 18 Jan 2024 01:47:44 GMT chrisf_sts 2024-01-18T01:47:44Z https://community.databricks.com/t5/get-started-discussions/how-to-handle-complex-json-schema/m-p/57635#M2252 <P>I have a mounted external directory that is an s3 bucket with multiple subdirectories containing call log files in json format.&nbsp; The files are irregular and complex, when i try to use spark.read.json or spark.sql (SELECT *) i get the&nbsp;UNABLE_TO_INFER_SCHEMA error.&nbsp; the files are too complex to try and build a schema manually, plus there are thousands of files.&nbsp; what is the best approach for creating a dataframe with this data?</P> Thu, 18 Jan 2024 01:47:44 GMT https://community.databricks.com/t5/get-started-discussions/how-to-handle-complex-json-schema/m-p/57635#M2252 chrisf_sts 2024-01-18T01:47:44Z