Empty xml tag

YevheniiY — Wed, 07 Feb 2024 14:03:03 GMT

Hello, I'm facing a problem with reading xml file via a new feature xml (auto) loader. Having similar structure and using rowTag as Item, reading is not done correctly. The data frame get only attributes, and the first line ItemID, further reading stops, although the tag is not closed and further there is still a lot of data. There are no errors, and does not say anything about incorrect operation. I tried different options for reading, removed attributes and so on. I used the native documentation https://docs.databricks.com/en/_extras/documents/native-xml-private-preview.pdf

Re: Empty xml tag

YevheniiY — Fri, 16 Feb 2024 14:05:26 GMT

@Retired_mod Hi! I apologise for the late reply! 😅 I'm using the 14.3 LTS (includes Apache Spark 3.5.0, Scala 2.12) version of the cluster. I use the python programming language. When I read other files that do not have empty tags (i.e. a tag that does not have an opening tag and a closing tag, but has a "merged" tag) like in my first post the tag "ItemId", the code works marvellously, but when it encounters an empty tag the reading stops. If the document consists of "normal" tags that have an opening and closing tag, the reading works fine! 🙂

And it's marvellous and makes it very easy to work with files in databricks, but empty tags don't work for me 😅

df = spark.read.format('xml').options(rowTag='Item').load(test_file_location) df.display()

Example for normal XML

Example of empty tag

topic Re: Empty xml tag in Data Engineering

Empty xml tag

Re: Empty xml tag