Hello @pjv ,

Greetings of the day!

The choice between having a worker node of the same type as your driver node or using a single-node compute cluster depends on the nature of your workload.

Suppose you are running normal Python code on a notebook as a job on this cluster, and your workload is not distributed (i.e., it doesn't require Spark's distributed computing capabilities). In that case, a single-node compute cluster might be sufficient. In a single-node compute cluster, the driver acts as both master and worker, with no worker nodes. This is intended for jobs that use small amounts of data or non-distributed workloads such as single-node machine learning libraries.
However, if your workload involves large-scale data processing or distributed machine learning tasks, having worker nodes of the same type as your driver node could improve performance. In a multi-node compute, worker nodes run the Spark executors and other services required for proper functioning compute. All the distributed processing happens on worker nodes.

I hope this clears your doubts!