topic Re: NATIVE_XML_DATA_SOURCE_NOT_ENABLED in Get Started Discussions

NATIVE_XML_DATA_SOURCE_NOT_ENABLED

Ajbi — Fri, 20 Oct 2023 17:00:08 GMT

I'm trying to read an xml file and receiving the following error. I've installed the maven library spark xml to the cluster, however I'm receiving the error. is there anything i'm missing?

Error

AnalysisException: [NATIVE_XML_DATA_SOURCE_NOT_ENABLED] Native XML Data Source is not enabled in this cluster.

runtime version: 14.1 (includes Apache Spark 3.5.0, Scala 2.12)

library version : com.databricks:spark-xml_2.12:0.17.0

code

from pyspark.sql import SparkSession

# Initialize Spark session
spark = SparkSession.builder.appName("XMLRead").getOrCreate()

# Path to the XML file
xml_path = '<path>'

# Read the XML file
df = spark.read \
.format("xml") \
.option("rootTag", "catalog") \
.option("rowTag", "book") \
.load(xml_path)

# Show the DataFrame
df.show()

Re: NATIVE_XML_DATA_SOURCE_NOT_ENABLED

daniel_sahal — Mon, 23 Oct 2023 05:31:24 GMT

@Ajbi AFAIK DBR 14.1 was supposed to support XML out of the box. Maybe that's the case?

Can you try using spark.read.format('com.databricks.spark.xml')... instead?

Re: NATIVE_XML_DATA_SOURCE_NOT_ENABLED

Ajbi — Mon, 23 Oct 2023 10:51:30 GMT

i've tried already spark.read.format('com.databricks.spark.xml'). it receives the same error.