Custom JobGroup in Spark UI for cluster with multiple executions

[FerArribas]

Does anyone know what the first digits of the jobgroup that are shown in the spark ui mean when using all purpose clusters to launch multiple jobs?

Right now the pattern is something like: [id_random]_job_[jod_id]_run-[run_id]_action_[action].

[saikumar246]

Hi @FerArribas 

The first digits of the jobgroup that are shown in the spark UI are execContextId and cmdId(Command_ID).

You can think of the execContextId as some kind of “REPL ID”

For example, if you take the below job group ID as an example,

jobGroupId: Option[String] = Some(3482599387390558128_6163657922384662685_job-532427916265395-run-416293225333467-action-6638250854918794)

execContextId = 3482599387390558128

cmdId = 6163657922384662685

executionId = job-532427916265395-run-416293225333467-action-6638250854918794