can we read XML files into Dataframes in Spark?

Srikanth_Gupta_
Databricks Employee
Databricks Employee
 

Ryan_Chynoweth
Databricks Employee
Databricks Employee

Yes, you can use the Spark XML library. https://github.com/databricks/spark-xml

Srikanth_Gupta_
Databricks Employee
Databricks Employee

Yes we can read using below code snippet

val df = spark.read .format("com.databricks.spark.xml").option("rowTag", "<message>") .load("sample.xml")

display(df)

rowTag is important to specify to read the actual content in XML

please read for more details

Mooune_DBU
Databricks Employee
Databricks Employee

Yes of course, you can use the

OR use

val df = spark.read
      .format("xml")
      .load("my_file.xml")

More info on the spark-xml api here

sean_owen
Databricks Employee
Databricks Employee

Note that you will need to install the spark-xml library to make this work: https://github.com/databricks/spark-xml For example you can create a Library in the workspace that references com.databricks:spark-xml_2.12:0.12.0 and then attach it to a cluster