For Server-less SQL Warehouses, Databricks claims that performance of such warehouses and the objects within is handled by Databricks itself. Which is believable based on the fact that Databricks still uses its Spark Engine to manipulate data in the SQL Warehouse (please correct me if I am wrong).
If the above statement is correct, then why do we need to think of all the query optimisations ? Table optimisations using OPTIMIZE and VACUUM (on delta tables) is still understandable. But query optimisation like using of CTE, Hints etc. ? Is that really necessary ? If yes, why ? What happened to the claims made for Spark SQL Engine and it's newfound sibling the AI ?
