https://community.databricks.com/t5/get-started-discussions/how-to-retrieve-a-job-name-from-the-sparkcontext/m-p/49784#M1633 <P>We are currently starting to build certain data pipelines using Databricks.</P><P>For this we use Jobs and the steps in these Jobs are implemented in Python Wheels.</P><P>We are able to retrieve the Job ID, Job Run ID and Task Run Id in our Python Wheels from the SparkContext using:</P><P>&nbsp;</P><DIV><DIV><DIV><DIV><DIV><PRE><SPAN>from </SPAN>pyspark.context <SPAN>import </SPAN>SparkContext<BR />sc = SparkContext.getOrCreate()<BR />sc.getLocalProperty(<SPAN>"spark.databricks.job.id"</SPAN>)<BR />sc.getLocalProperty(<SPAN>"spark.databricks.job.parentRunId"</SPAN>)<BR />sc.getLocalProperty(<SPAN>"spark.databricks.job.runId"</SPAN>)</PRE></DIV>&nbsp;</DIV></DIV></DIV><DIV><SPAN>Now we would also like to get the name of the current Job which is running the wheel. Is there any local property containing that information? So far we did not manage to retrieve all available Keys which are stored as local Properties. Is there any documentation describing all available Properties? We did not find anything out there...</SPAN></DIV><DIV>&nbsp;</DIV><DIV><SPAN>Thanks in advance for your advice!</SPAN></DIV></DIV> Tue, 24 Oct 2023 06:42:44 GMT HHol 2023-10-24T06:42:44Z https://community.databricks.com/t5/get-started-discussions/how-to-retrieve-a-job-name-from-the-sparkcontext/m-p/49784#M1633 <P>We are currently starting to build certain data pipelines using Databricks.</P><P>For this we use Jobs and the steps in these Jobs are implemented in Python Wheels.</P><P>We are able to retrieve the Job ID, Job Run ID and Task Run Id in our Python Wheels from the SparkContext using:</P><P>&nbsp;</P><DIV><DIV><DIV><DIV><DIV><PRE><SPAN>from </SPAN>pyspark.context <SPAN>import </SPAN>SparkContext<BR />sc = SparkContext.getOrCreate()<BR />sc.getLocalProperty(<SPAN>"spark.databricks.job.id"</SPAN>)<BR />sc.getLocalProperty(<SPAN>"spark.databricks.job.parentRunId"</SPAN>)<BR />sc.getLocalProperty(<SPAN>"spark.databricks.job.runId"</SPAN>)</PRE></DIV>&nbsp;</DIV></DIV></DIV><DIV><SPAN>Now we would also like to get the name of the current Job which is running the wheel. Is there any local property containing that information? So far we did not manage to retrieve all available Keys which are stored as local Properties. Is there any documentation describing all available Properties? We did not find anything out there...</SPAN></DIV><DIV>&nbsp;</DIV><DIV><SPAN>Thanks in advance for your advice!</SPAN></DIV></DIV> Tue, 24 Oct 2023 06:42:44 GMT https://community.databricks.com/t5/get-started-discussions/how-to-retrieve-a-job-name-from-the-sparkcontext/m-p/49784#M1633 HHol 2023-10-24T06:42:44Z