Empty xml tag

[YevheniiY]

<ItemMaintenance>
	<Batch>
	<BathInfo>info</BathInfo>
		<Item attr1="tekst" attr2="Tekst2">
			<ItemId type="Type" id="id"/>
			<Dates>
				<Start>2023-11-09</Start>
				<End>2024-01-02</End>
			</Dates>
			<MoreData>
			More data
			</MoreData>
		</Item>
	</Batch>
</ItemMaintenance>

Hello, I'm facing a problem with reading xml file via a new feature xml (auto) loader. Having similar structure and using rowTag as Item, reading is not done correctly. The data frame get only attributes, and the first line ItemID, further reading stops, although the tag is not closed and further there is still a lot of data. There are no errors, and does not say anything about incorrect operation. I tried different options for reading, removed attributes and so on. I used the native documentation https://docs.databricks.com/en/_extras/documents/native-xml-private-preview.pdf

[YevheniiY]

@Retired_mod Hi! I apologise for the late reply! 😅 I'm using the 14.3 LTS (includes Apache Spark 3.5.0, Scala 2.12) version of the cluster. I use the python programming language. When I read other files that do not have empty tags (i.e. a tag that does not have an opening tag and a closing tag, but has a "merged" tag) like in my first post the tag "ItemId", the code works marvellously, but when it encounters an empty tag the reading stops. If the document consists of "normal" tags that have an opening and closing tag, the reading works fine! 🙂

And it's marvellous and makes it very easy to work with files in databricks, but empty tags don't work for me 😅

df = spark.read.format('xml').options(rowTag='Item').load(test_file_location)
df.display()

Example for normal XML

  <people>
    <person>
      <age born="1990-02-24">25</age>
    </person>
    <person>
      <age born="1985-01-01">30</age>
    </person>
    <person>
      <age born="1980-01-01">30</age>
    </person>
  </people>

Example of empty tag

  <people>
    <person>
      <age_t born="1990-02-24"/>
      <age born="1990-02-24">25</age>
    </person>
    <person>
      <age_t born="1985-01-01"/>
      <age born="1985-01-01">30</age>
    </person>
    <person>
     <age_t born="1980-01-01"/>
      <age born="1980-01-01">30</age>
    </person>
  </people>