Your subscription deserves a post and a video 😉 to clarify how to leverage caching stuff with "Databricks Disk Caching" (provided by DBR) and "Spark Caching" (provided by Spark) working together. In a nutshell it is as you can see below:

Both can work as a L1/L2 cache in cluster worker nodes only if cloud VMs have SSDs (Solid Stage Drives). Otherwise, only Apache Spark Caching can be used. I'll explain later in detail, but take a look at picture below to summarize features of both caching types:

I hope this helps!