I have installed graphframes library from maven repository in the cluster (13.3 LTS (includes Apache Spark 3.4.1, Scala 2.12), Standard DS4_v2). The library that I have installed is graphframes:graphframes:0.8.3-spark3.5-s_2.13.
I can import the graphframes library in the notebook as well but running the following command is giving me error -
g = GraphFrame(vertices, edges)
Error -
java.lang.NoClassDefFoundError: scala/collection/ArrayOps$
---------------------------------------------------------------------------
Py4JJavaError Traceback (most recent call last)
Py4JJavaError: An error occurred while calling o16831.createGraph. : java.lang.NoClassDefFoundError: scala/collection/ArrayOps$
How can I rectify this?
Thanks!
