Data engineering professional exam

[anshi_t_k]

Each configuration below is identical in that each cluster has 400 GB total of RAM, 160 total cores, and only one Executor per VM.

Given an extremely long-running job for which completion must be guaranteed, which cluster configuration will be able to guarantee completion of the job in light of one or more VM failures?

A. Total VMs: 8, 50 GB per Executor, 20 Cores / Executor
B. Total VMs: 16, 25 GB per Executor, 10 Cores / Executor
C. Total VMs: 1, 400 GB per Executor, 160 Cores / Executor
D. Total VMs: 4, 100 GB per Executor, 40 Cores / Executor
E. Total VMs: 2, 200 GB per Executor, 80 Cores / Executor

[anshi_t_k]

I am confused between option B and D according to different sites, can anyone provide a clarity regarding this

[filipniziol]

Hi @anshi_t_k ,

The key consideration here is fault tolerance. 
How do you protect against a VM failure? By having more VMs, as the impact of a single VM  failure will be the lowest.

For example answer C - the crash of the VM is loosing 1/1 so 100% capacity: no fault tolerance.
In case of answer B - the crash of the VM means loosing 1/16, so only 6,25% capacity.
The other answer you mentioned B - the crash of the VM means loosing 1/4, so 25% capacity.

The answer is B:

• It has the highest number of VMs, which spreads the workload more evenly.
• The impact of any single VM failure is minimal, with only 6.25% of the total capacity lost per VM failure.
• This configuration provides the best fault tolerance, as the cluster can sustain multiple VM failures and still continue to operate effectively, ensuring that the job can complete.

[anshi_t_k]

Thank you for the information.