Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-16-2022 08:34 PM
In Spark we can get the Spark Application ID inside the Task programmatically using:
SparkEnv.get.blockManager.conf.getAppIdand we can get the Stage ID and Task Attempt ID of the running Task using:
TaskContext.get.stageId
TaskContext.get.taskAttemptIdIs there any way to get the Spark Job Id that is associated with a running Task (preferably using TaskContext or SparkEnv)?
Linked Question on StackOverflow: https://stackoverflow.com/questions/70929032/how-to-programmatically-get-the-spark-job-id-of-a-runni...
Labels:
- Labels:
-
Spark job