Etyr
Contributor II

I managed to recreate the error in compute instance, by deleting the /etc/hosts file.

Exception in thread "main" java.lang.ExceptionInInitializerError
        at org.apache.spark.deploy.SparkSubmitArguments.$anonfun$loadEnvironmentArguments$5(SparkSubmitArguments.scala:163)
        at scala.Option.orElse(Option.scala:447)
        at org.apache.spark.deploy.SparkSubmitArguments.loadEnvironmentArguments(SparkSubmitArguments.scala:163)
        at org.apache.spark.deploy.SparkSubmitArguments.<init>(SparkSubmitArguments.scala:118)
        at org.apache.spark.deploy.SparkSubmit$$anon$2$$anon$3.<init>(SparkSubmit.scala:1046)
        at org.apache.spark.deploy.SparkSubmit$$anon$2.parseArguments(SparkSubmit.scala:1046)
        at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:85)
        at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1063)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1072)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.net.UnknownHostException: my-compute-instance: my-compute-instance: Name or service not known
        at java.net.InetAddress.getLocalHost(InetAddress.java:1432)
        at org.apache.spark.util.Utils$.findLocalInetAddress(Utils.scala:1211)
        at org.apache.spark.util.Utils$.localIpAddress$lzycompute(Utils.scala:1204)
        at org.apache.spark.util.Utils$.localIpAddress(Utils.scala:1204)
        at org.apache.spark.util.Utils$.$anonfun$localCanonicalHostName$1(Utils.scala:1261)
        at scala.Option.getOrElse(Option.scala:189)
        at org.apache.spark.util.Utils$.localCanonicalHostName(Utils.scala:1261)
        at org.apache.spark.internal.config.package$.<init>(package.scala:1080)
        at org.apache.spark.internal.config.package$.<clinit>(package.scala)
        ... 10 more
Caused by: java.net.UnknownHostException: my-compute-instance: Name or service not known
        at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
        at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:867)
        at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1302)
        at java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:815)
        at java.net.InetAddress.getAllByName0(InetAddress.java:1291)
        at java.net.InetAddress.getLocalHost(InetAddress.java:1427)
        ... 18 more

Here is the file in the compute instance:

127.0.0.1 localhost my-compute-instance my-compute-instance my-compute-instance my-compute-instance my-compute-instance my-compute-instance my-compute-instance my-compute-instance my-compute-instance my-compute-instance my-compute-instance my-compute-instance my-compute-instance my-compute-instance

# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

Here is the hosts file in compute clusters:

127.0.0.1 localhost

# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts