- 28658 Views
- 3 replies
- 4 kudos
We are having Databricks Job running with main class and JAR file in it. Our JAR file code base is in Scala. Now, when our job starts running, we need to log Job ID and Run ID into the database for future purpose. How can we achieve this?
- 28658 Views
- 3 replies
- 4 kudos
Latest Reply
That article is for members only, can we also specify here how to do it (for those that are not medium members?). Thanks!
2 More Replies
- 6759 Views
- 2 replies
- 1 kudos
The Databricks widget (dbutils) provides the get function for accessing the job parameters of a job.dbutils.widgets.get('my_param')Unlike Python dict, where get returns None or an optional argument if the dict doesn't contain the parameter, the widg...
- 6759 Views
- 2 replies
- 1 kudos
Latest Reply
Hi @Mattias P Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers you...
1 More Replies
- 3241 Views
- 1 replies
- 1 kudos
In Databricks jobs on Azure you can use the {{run_id}} and {{parent_run_id}}variables for a specific run: https://docs.databricks.com/workflows/jobs/jobs.htmlFor Databricks jobs with only two or more tasks, then {{run_id}} seems to correspond to task...
- 3241 Views
- 1 replies
- 1 kudos
Latest Reply
@Kasper H :Yes, you are correct in your understanding that in Databricks jobs with multiple tasks, the {{run_id}} variable corresponds to the task_run_id and the {{parent_run_id}} variable corresponds to the job_run_id.For Databricks jobs with only ...
- 6903 Views
- 3 replies
- 3 kudos
Can I run multiple jobs(for example: 100+) in parallel that refers the same notebook? I supply each job with a different parameter. If we can do this, what would be the impact? (for example: reliability, performance, troubleshooting etc. )Example: N...
- 6903 Views
- 3 replies
- 3 kudos
Latest Reply
Hi @Murthy Ramalingam Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you....
2 More Replies
by
rammy
• Contributor III
- 9575 Views
- 5 replies
- 5 kudos
I have tried following ways to get job parameters but none of the things are working.runId='{{run_id}}'
jobId='{{job_id}}'
filepath='{{filepath}}'
print(runId," ",jobId," ",filepath)
r1=dbutils.widgets.get('{{run_id}}')
f1=dbutils.widgets.get('{{file...
- 9575 Views
- 5 replies
- 5 kudos
Latest Reply
Thanks for your response. I found the solution. The below code gives me all the job parametersall_args = dbutils.notebook.entry_point.getCurrentBindings()print(all_args)Thanks for your support
4 More Replies
- 2372 Views
- 3 replies
- 4 kudos
I have a notebook that sets up parameters for the run based on some job parameters set by the user as well as the current date of the run. I want to supersede some of this logic and just use the manual values if kicked off manually. Is there a way to...
- 2372 Views
- 3 replies
- 4 kudos
Latest Reply
You can create widgets by using this- dbutils.widgets.text("widgetName", "")To get the value for that widget:- dbutils.widgets.get("widgetName")So by using this you can manually create widgets (variable) and can run the process by giving desired valu...
2 More Replies
