About your third question, you can get to know the actual value of spark.driver.memory by looking at the executors tab in spark UI. This will also have the driver and we can get to know the actual value of the driver memory.
In the spark UI, we would be able to see only the storage memory. Execution memory will almost be equal to the storage memory.
In this case, driver memory would be 21.4GB. This will be the amount of memory allocated to JVM related activities.
