How to run commands on the executor

[brickster_2018]

Using %sh, I am able to run commands on the notebook and get output. How can i run a command on the executor and get the output. I want to avoid using the Spark API's

[brickster_2018 · Accepted solution]

It's not possible to use %sh to run commands on the executor. The below code can be used to run commands on the executor and get the output

var res=sc.runOnEachExecutor[String]({ () =>
import sys.process._
  
       var cmd_Result=Seq("bash", "-c", "hostname").!!
      cmd_Result
    }, 100.seconds)