- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-20-2024 06:15 AM
When using Asset Bundles to deploy jobs, how does one get the job ID of the resources that are created?
I would like to deploy some jobs through asset bundles, get the job IDs, and then trigger these jobs programmatically outside the CI/CD pipeline using the rest API in Python. Though the output of the databricks bundle deploy command does not seem to include the ID of the jobs that are created so I can't pass them to subsequent steps / other systems.
I understand that I can get the job ID using the UI but I would like to orchestrate my CI/CD without the need to provide a manual input.
Thank you in advance!
- Labels:
-
Workflows
Accepted Solutions
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-20-2024 09:46 AM
Hey, not sure if this will do the trick, but i've thought about two workarounds:
1. Check if the "databricks bundle run my_job" suits your case. It accepts the name as the key to run here.
2. Would it be an option for you to use databricks jobs list command (you can also filter with job names) and extract the job id?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-20-2024 09:46 AM
Hey, not sure if this will do the trick, but i've thought about two workarounds:
1. Check if the "databricks bundle run my_job" suits your case. It accepts the name as the key to run here.
2. Would it be an option for you to use databricks jobs list command (you can also filter with job names) and extract the job id?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-20-2024 08:50 AM
Thanks @mhiltner. I don't need to run jobs, just to get the ID. So I think that solution 2) is the way to go here. I'll accept the solution.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
a month ago
Refer this answer and this can be a solution to above scenario -> https://community.databricks.com/t5/data-engineering/getting-job-id-dynamically-to-create-another-jo...