Custom JobGroup in Spark UI for cluster with multiple executions

[FerArribas]

Does anyone know what the first digits of the jobgroup that are shown in the spark ui mean when using all purpose clusters to launch multiple jobs?

Right now the pattern is something like: [id_random]_job_[jod_id]_run-[run_id]_action_[action].

[saikumar246]

Hi @FerArribas 

The first digits of the jobgroup that are shown in the spark UI are execContextId and cmdId(Command_ID).

You can think of the execContextId as some kind of “REPL ID”

For example, if you take the below job group ID as an example,

jobGroupId: Option[String] = Some(3482599387390558128_6163657922384662685_job-532427916265395-run-416293225333467-action-6638250854918794)

execContextId = 3482599387390558128

cmdId = 6163657922384662685

executionId = job-532427916265395-run-416293225333467-action-6638250854918794

[Kaniz_Fatma]

Hey there! Thanks a bunch for being part of our awesome community! 🎉 

We love having you around and appreciate all your questions. Take a moment to check out the responses – you'll find some great info. Your input is valuable, so pick the best solution for you. And remember, if you ever need more help, we're here for you! 

Keep being awesome! 😊🚀