Adding these options
EXTRA_JAVA_OPTIONS = (
'-Dcom.sun.management.jmxremote.port=9999',
'-Dcom.sun.management.jmxremote.authenticate=false',
'-Dcom.sun.management.jmxremote.ssl=false',
)
is enough in vanilla Apache Spark, but apparently it is not enough in Databricks. How to I enable it so as to use VisualVM/jconsole to debug performance bottlenecks?
Sergey