Hello Anudeep,

Thank you for your detailed response and the helpful recommendations.

I would like to provide some additional context:

1. For our jobs, we are running only one notebook at a time, not multiple notebooks or tasks concurrently.
2. The issue occurs before the notebook execution starts — we do not see any cell execution in the logs.
3. We also could not find any error message like "Unable to start python kernel for ReplId-<id>, kernel did not start within 80 seconds."
4. This error happens right at the start of a job run using a job cluster, before any code in the notebook is executed.

Regarding your suggestion about changing to a memory-optimized driver series, thank you for the recommendation — we will definitely consider this option.

Please let me know if there are any additional logs or metrics you would recommend checking in this specific scenario.

Thanks & Regards,
Sandeep