Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-04-2023 08:16 AM
Unfortunately, running multiple jobs in parallel using a single job cluster is not supported (yet). New in databricks is the possibility to create a job that orchestrates multiple jobs. These jobs will however still use their own cluster (configuration).
In the case the time of instancing a cluster takes much longer than executing an actual job, you could consider using `all-purpose` cluster. Depending on your use-case it might also be possible to rewrite/re-configure your 5 jobs to a single job with 5 tasks.