I think you can get some pretty good insight into the optimizations on Databricks here:

Specifically, check out the sections on caching, z-ordering, and join optimization. There's also a great, detailed blog post here: Processing Petabytes of Data in Seconds with Databricks Delta

Hope this helps! @kali.tummala@gmail.com