cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
cancel
Showing results for 
Search instead for 
Did you mean: 

NATIVE_XML_DATA_SOURCE_NOT_ENABLED

Ajbi
New Contributor II

I'm trying to read an xml file and receiving the following error. I've installed the maven library spark xml to the cluster, however I'm receiving the error. is there anything i'm missing?

Error

AnalysisException: [NATIVE_XML_DATA_SOURCE_NOT_ENABLED] Native XML Data Source is not enabled in this cluster.

runtime version: 14.1 (includes Apache Spark 3.5.0, Scala 2.12)

library version : com.databricks:spark-xml_2.12:0.17.0

code

 

from pyspark.sql import SparkSession

# Initialize Spark session
spark = SparkSession.builder.appName("XMLRead").getOrCreate()

# Path to the XML file
xml_path = '<path>'

# Read the XML file
df = spark.read \
    .format("xml") \
    .option("rootTag", "catalog") \
    .option("rowTag", "book") \
    .load(xml_path)

# Show the DataFrame
df.show()

3 REPLIES 3

daniel_sahal
Honored Contributor III

@Ajbi AFAIK DBR 14.1 was supposed to support XML out of the box. Maybe that's the case?

 

Can you try using spark.read.format('com.databricks.spark.xml')... instead?

Ajbi
New Contributor II

i've tried already  spark.read.format('com.databricks.spark.xml'). it receives the same error. 

 

dbruehlmeier
Contributor

@Ajbi I run into same issue with DBR 14.1 and "Single user" mode cluster.

Possible workarounds: 

  • Use DBR 14.1, "Shared" access mode, no library installed and Spark config: spark.databricks.connector.enableXml true
  • Use DBR 14.0, "Single user" access mode, install library: com.databricks:spark-xml_2.12:0.17.0

@Kaniz: Why does DBR 14.1 with "Single user" mode not work?

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.