- 9795 Views
- 2 replies
- 12 kudos
How concurrent runs in a job matches to cluster configuration?
In databricks jobs, there's a field to add concurrent runs which can be set to 1000.If I've a cluster with 4 worker nodes and 8 cores each, then at max how many concurrent jobs I'll be able to execute?What will happen if I launch 100 instances of sam...
- 9795 Views
- 2 replies
- 12 kudos
Latest Reply
- 12 kudos
@Aman Sehgal​ On E2 workspace the limit is 1000 concurrent runs. If you trigger 100 runs​ at the same time, 100 clusters will be created and the runs will be executed. If you use the same cluster for 100 runs, then you might face a lot of failed jobs...
- 12 kudos
