Workaround for GraphFrames not working on Delta Live Table?
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-28-2022 05:22 PM
According to this page, the GraphFrames package is included in the databricks runtime since at least 11.0. However trying to run a connected components algorithm inside a delta live table notebook yields the error java.lang.ClassNotFoundException: org.graphframes.GraphFramePythonAPI
I installed with pip using a magic command, but it seems that the package is not included in the cluster itself. Is there a workaround to get graphframes working on the delta live runtime? I tried adding the maven coordinates in the cluster definition but it seems DLT does not support maven libraries.
Labels:
- Labels:
-
Databricks Runtime
-
DLT
-
Graphframes