If only the driver is active, this probably means you are not using Spark.  When running pure python,... code, the driver will execute that.
If Spark is active, workers receive their tasks from the driver.  Generally the driver is not that active, the workers do all the work.The driver machine is not designed in any way.  You can define yourself what kind of machine you use as a driver.
You can even run in single node mode, so you only have a driver (which also acts as a worker).