Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-30-2024 06:25 AM
I managed to recreate the error in compute instance, by deleting the /etc/hosts file.
Exception in thread "main" java.lang.ExceptionInInitializerError
at org.apache.spark.deploy.SparkSubmitArguments.$anonfun$loadEnvironmentArguments$5(SparkSubmitArguments.scala:163)
at scala.Option.orElse(Option.scala:447)
at org.apache.spark.deploy.SparkSubmitArguments.loadEnvironmentArguments(SparkSubmitArguments.scala:163)
at org.apache.spark.deploy.SparkSubmitArguments.<init>(SparkSubmitArguments.scala:118)
at org.apache.spark.deploy.SparkSubmit$$anon$2$$anon$3.<init>(SparkSubmit.scala:1046)
at org.apache.spark.deploy.SparkSubmit$$anon$2.parseArguments(SparkSubmit.scala:1046)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:85)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1063)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1072)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.net.UnknownHostException: my-compute-instance: my-compute-instance: Name or service not known
at java.net.InetAddress.getLocalHost(InetAddress.java:1432)
at org.apache.spark.util.Utils$.findLocalInetAddress(Utils.scala:1211)
at org.apache.spark.util.Utils$.localIpAddress$lzycompute(Utils.scala:1204)
at org.apache.spark.util.Utils$.localIpAddress(Utils.scala:1204)
at org.apache.spark.util.Utils$.$anonfun$localCanonicalHostName$1(Utils.scala:1261)
at scala.Option.getOrElse(Option.scala:189)
at org.apache.spark.util.Utils$.localCanonicalHostName(Utils.scala:1261)
at org.apache.spark.internal.config.package$.<init>(package.scala:1080)
at org.apache.spark.internal.config.package$.<clinit>(package.scala)
... 10 more
Caused by: java.net.UnknownHostException: my-compute-instance: Name or service not known
at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:867)
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1302)
at java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:815)
at java.net.InetAddress.getAllByName0(InetAddress.java:1291)
at java.net.InetAddress.getLocalHost(InetAddress.java:1427)
... 18 moreHere is the file in the compute instance:
127.0.0.1 localhost my-compute-instance my-compute-instance my-compute-instance my-compute-instance my-compute-instance my-compute-instance my-compute-instance my-compute-instance my-compute-instance my-compute-instance my-compute-instance my-compute-instance my-compute-instance my-compute-instance
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhostsHere is the hosts file in compute clusters:
127.0.0.1 localhost
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts