โ12-15-2022 09:29 AM
I have tried following ways to get job parameters but none of the things are working.
runId='{{run_id}}'
jobId='{{job_id}}'
filepath='{{filepath}}'
print(runId," ",jobId," ",filepath)
r1=dbutils.widgets.get('{{run_id}}')
f1=dbutils.widgets.get('{{filepath}}')
print(r1,f1)
any suggestions to get those parameters is really helpful
โ12-15-2022 12:50 PM
Hi @Ramesh Bathiniโ , have you tried using dbutils? try this,
%scala
print (dbutils.notebook.getContext().jobId)
print (dbutils.notebook.getContext().currentRunId)
or
%scala
val jobId = sc.getLocalProperty("spark.databricks.job.id")
val runId = sc.getLocalProperty("spark.databricks.job.runId")
print("jobId")
print("runId")
โ12-15-2022 09:51 PM
Thank you @Vivian Wilfredโ for your response. As you said i could read runId and JobId values but how can i read job parameters?
In above question, filepath is a job parameter that takes file path and that should be able to access inside the notebook. Which library can help me to read that?
โ12-16-2022 10:22 AM
@Ramesh Bathiniโ You mean the notebook file path that the job runs? For notebook file path you can try this:
%scala print(dbutils.notebook.getContext().extraContext("notebook_path"))
dbutils.notebook.getContext() has all the job parameters, You can query the output of this command to get the job details that you are looking for,
Let me know if this helps. Mark the comment as "best answer" to resolve the query. Thanks.
โ12-15-2022 10:55 PM
Hi @Ramesh Bathiniโ you can use below code -
curl "https://centralus.azuredatabricks.net/api/2.0/jobs/runs/list" -X GET -H "Authorization: Bearer dapia08sjflksjs9jfra6a34a"
โ12-15-2022 11:44 PM
Hi @Ramesh Bathiniโ, We havenโt heard from you since the last response from @Ajay Pandeyโ and I was checking back to see if his suggestions helped you.
Or else, If you have any solution, please do share that with the community as it can be helpful to others.
Also, Please don't forget to click on the "Select As Best" button whenever the information provided helps resolve your question.
โ12-17-2022 09:17 AM
Thanks for your response. I found the solution. The below code gives me all the job parameters
all_args = dbutils.notebook.entry_point.getCurrentBindings()
print(all_args)
Thanks for your support
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
Click here to register and join today!
Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.