cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

How I could read the Job id, run id and parameters in python cell?

rammy
Contributor III

I have tried following ways to get job parameters but none of the things are working.

runId='{{run_id}}'
jobId='{{job_id}}'
filepath='{{filepath}}'
print(runId," ",jobId," ",filepath)
r1=dbutils.widgets.get('{{run_id}}')
f1=dbutils.widgets.get('{{filepath}}')
print(r1,f1)

any suggestions to get those parameters is really helpful

6 REPLIES 6

Vivian_Wilfred
Honored Contributor
Honored Contributor

Hi @Ramesh Bathini​ , have you tried using dbutils? try this,

%scala

print (dbutils.notebook.getContext().jobId)

print (dbutils.notebook.getContext().currentRunId)

or

%scala

val jobId = sc.getLocalProperty("spark.databricks.job.id")

val runId = sc.getLocalProperty("spark.databricks.job.runId")

print("jobId")

print("runId")

rammy
Contributor III

Thank you @Vivian Wilfred​  for your response. As you said i could read runId and JobId values but how can i read job parameters?

In above question, filepath is a job parameter that takes file path and that should be able to access inside the notebook. Which library can help me to read that?

Vivian_Wilfred
Honored Contributor
Honored Contributor

@Ramesh Bathini​ You mean the notebook file path that the job runs? For notebook file path you can try this:

%scala print(dbutils.notebook.getContext().extraContext("notebook_path"))

dbutils.notebook.getContext() has all the job parameters, You can query the output of this command to get the job details that you are looking for,

Let me know if this helps. Mark the comment as "best answer" to resolve the query. Thanks.

Ajay-Pandey
Esteemed Contributor III

Hi @Ramesh Bathini​ you can use below code -

curl "https://centralus.azuredatabricks.net/api/2.0/jobs/runs/list" -X GET -H "Authorization: Bearer dapia08sjflksjs9jfra6a34a"

Kaniz_Fatma
Community Manager
Community Manager

Hi @Ramesh Bathini​, We haven’t heard from you since the last response from @Ajay Pandey​ and I was checking back to see if his suggestions helped you.

Or else, If you have any solution, please do share that with the community as it can be helpful to others.

Also, Please don't forget to click on the "Select As Best" button whenever the information provided helps resolve your question.

rammy
Contributor III

Thanks for your response. I found the solution. The below code gives me all the job parameters

all_args = dbutils.notebook.entry_point.getCurrentBindings()

print(all_args)

Thanks for your support

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!