10-13-2021 07:22 AM
I've created an external table that points to an XML file on the ADLS. I've installed spark-xml library on my cluster. I can query the table on the cluster.
However, I cannot query the table on Databricks SQL. I get the following error. I suppose the spark-xml library should also be installed on the SQL endpoint, however there is no option to do so. Is there any way to install the library on the end point ?
java.util.concurrent.ExecutionException: java.lang.ClassNotFoundException: Failed to find data source: com.databricks.spark.xml. Please find packages at http://spark.apache.org/third-party'. (35) (SQLExecDirectW)
10-13-2021 08:48 AM
10-14-2021 12:07 AM
Yes. on the Cluster, however the SQL endpoint (cluster for Databricks SQL) does not offer this option.
10-13-2021 09:23 AM
Hello @Bilal Haniff - My name is Piper and I'm a moderator for the community. It's nice to meet you!
Let's see how your fellow community members answer first and then the team will follow up if they need to.
10-15-2021 04:23 AM
@Piper Wilson I believe this is a technical limitation of the platform (Databricks SQL). Can you please check with the team if this is the case?
10-15-2021 09:26 AM
@Bilal Haniff - Yes. It may be next week before I can get back to you. Thank you for your patience.
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group