Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-09-2024 10:15 PM
Here is one of the ways to use the struct field name as rowTag:
import org.apache.spark.sql.types._
val schema = new StructType().add("Record",
new StructType().add("age", IntegerType).add("name", StringType))
val data = Seq(Row(Row(18, "John Doe")), Row(Row(19, "Mary Doe")))
val df = spark.createDataFrame(spark.sparkContext.parallelize(data), schema)
val rowTag = schema.fields.head.name
df.coalesce(1).select(s"$rowTag.*").write.mode("Overwrite").option("rowTag", rowTag).xml("/tmp/xml_test")
If the generated XML file above read again, it will have a flattened schema with two fields ('age' and 'name') instead of a single struct column.