topic Re: What types of files does autoloader support for streaming ingestion ? I see good support for CSV and JSON, how can I ingest files like XML, avro, parquet etc ? would XML rely on Spark-XML ? in Data Engineering https://community.databricks.com/t5/data-engineering/what-types-of-files-does-autoloader-support-for-streaming/m-p/20322#M13706 <P>Please raise a feature request via <A href="https://docs.databricks.com/resources/ideas.html" alt="https://docs.databricks.com/resources/ideas.html" target="_blank">ideas</A> portal for XML support in autoloader </P><P></P><P>As a workaround, you could look at reading this with <A href="https://spark.apache.org/docs/latest/api/java/org/apache/spark/SparkContext.html#wholeTextFiles-java.lang.String-int-" alt="https://spark.apache.org/docs/latest/api/java/org/apache/spark/SparkContext.html#wholeTextFiles-java.lang.String-int-" target="_blank">wholeTextFiles</A> (which loads the data into a PairRDD with one record per input file) and parsing it with <A href="https://github.com/databricks/spark-xml/blob/2ba6736a385bd5dc8e18d64aedb1d82891d227dc/README.md#parsing-nested-xml" alt="https://github.com/databricks/spark-xml/blob/2ba6736a385bd5dc8e18d64aedb1d82891d227dc/README.md#parsing-nested-xml" target="_blank">from_xml</A> from the spark-xml package </P> Fri, 25 Jun 2021 00:26:26 GMT sajith_appukutt 2021-06-25T00:26:26Z What types of files does autoloader support for streaming ingestion ? I see good support for CSV and JSON, how can I ingest files like XML, avro, parquet etc ? would XML rely on Spark-XML ? https://community.databricks.com/t5/data-engineering/what-types-of-files-does-autoloader-support-for-streaming/m-p/20321#M13705 <P>What types of files does autoloader support for streaming ingestion ? I see good support for CSV and JSON, how can I ingest files like XML, avro, parquet etc ? would XML rely on Spark-XML ?&nbsp;</P> Thu, 24 Jun 2021 21:07:43 GMT https://community.databricks.com/t5/data-engineering/what-types-of-files-does-autoloader-support-for-streaming/m-p/20321#M13705 User16783853501 2021-06-24T21:07:43Z Re: What types of files does autoloader support for streaming ingestion ? I see good support for CSV and JSON, how can I ingest files like XML, avro, parquet etc ? would XML rely on Spark-XML ? https://community.databricks.com/t5/data-engineering/what-types-of-files-does-autoloader-support-for-streaming/m-p/20322#M13706 <P>Please raise a feature request via <A href="https://docs.databricks.com/resources/ideas.html" alt="https://docs.databricks.com/resources/ideas.html" target="_blank">ideas</A> portal for XML support in autoloader </P><P></P><P>As a workaround, you could look at reading this with <A href="https://spark.apache.org/docs/latest/api/java/org/apache/spark/SparkContext.html#wholeTextFiles-java.lang.String-int-" alt="https://spark.apache.org/docs/latest/api/java/org/apache/spark/SparkContext.html#wholeTextFiles-java.lang.String-int-" target="_blank">wholeTextFiles</A> (which loads the data into a PairRDD with one record per input file) and parsing it with <A href="https://github.com/databricks/spark-xml/blob/2ba6736a385bd5dc8e18d64aedb1d82891d227dc/README.md#parsing-nested-xml" alt="https://github.com/databricks/spark-xml/blob/2ba6736a385bd5dc8e18d64aedb1d82891d227dc/README.md#parsing-nested-xml" target="_blank">from_xml</A> from the spark-xml package </P> Fri, 25 Jun 2021 00:26:26 GMT https://community.databricks.com/t5/data-engineering/what-types-of-files-does-autoloader-support-for-streaming/m-p/20322#M13706 sajith_appukutt 2021-06-25T00:26:26Z