Data engineering professional exam

anshi_t_k
New Contributor III

‎09-03-2024 03:07 AM

Each configuration below is identical in that each cluster has 400 GB total of RAM, 160 total cores, and only one Executor per VM.

Given an extremely long-running job for which completion must be guaranteed, which cluster configuration will be able to guarantee completion of the job in light of one or more VM failures?

A. Total VMs: 8, 50 GB per Executor, 20 Cores / Executor
B. Total VMs: 16, 25 GB per Executor, 10 Cores / Executor
C. Total VMs: 1, 400 GB per Executor, 160 Cores / Executor
D. Total VMs: 4, 100 GB per Executor, 40 Cores / Executor
E. Total VMs: 2, 200 GB per Executor, 80 Cores / Executor

0 Kudos

anshi_t_k
New Contributor III

‎09-03-2024 03:09 AM

I am confused between option B and D according to different sites, can anyone provide a clarity regarding this

 

0 Kudos

filipniziol
Esteemed Contributor

‎09-03-2024 03:31 AM - edited ‎09-03-2024 03:32 AM

Hi @anshi_t_k ,

The key consideration here is fault tolerance. 
How do you protect against a VM failure? By having more VMs, as the impact of a single VM  failure will be the lowest.

For example answer C - the crash of the VM is loosing 1/1 so 100% capacity: no fault tolerance.
In case of answer B - the crash of the VM means loosing 1/16, so only 6,25% capacity.
The other answer you mentioned B - the crash of the VM means loosing 1/4, so 25% capacity.

The answer is B:

  • It has the highest number of VMs, which spreads the workload more evenly.
  • The impact of any single VM failure is minimal, with only 6.25% of the total capacity lost per VM failure.
  • This configuration provides the best fault tolerance, as the cluster can sustain multiple VM failures and still continue to operate effectively, ensuring that the job can complete.

anshi_t_k
New Contributor III
In response to filipniziol

‎09-03-2024 04:31 AM

Thank you for the information.

0 Kudos