Louis_Frolio
Databricks Employee
Databricks Employee

If you want to know more about query optimization I suggest you look in Spark's Catalyst Optimizer and Adaptive Query Execution (AQE).  You should alway run Analyze table compute statistics becuase this will help Spark's query optimization converge on the right execution plan via an Optimized Logical plan and then the Optimized Physial plan.  There is plenty of information out there if you really want to dig in and learn.  But the short of it is that Delta Statisics and Spark Query Optimizer do work hand in hand but there are things about Delta Statistics that only relate to Delta. Hope this help. Lou.