Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-13-2022 02:54 AM
Hi @Salah K. - I am sorry for the confusion. I mean to say use bigger cluster and verify. Something like below.
Standard_M8ms 3
vCPU = 8
Memory: GiB = 218
https://docs.microsoft.com/en-us/azure/virtual-machines/m-series
I am not that familiar with Azure. I am also doing xml parsing in AWS workspace. But the files I am loading are not this huge.
val df = spark.
read.format("com.databricks.spark.xml")
.option("rowTag", "<MyRowTag>")
.option("rootTag", "<MyRoootTag>")
.load("<XML File Path>")
Turn off the below option, if it is true.
.option("inferschema", "false")