cgrant
Databricks Employee
Databricks Employee

For this kind of table maintenance, Predictive Optimization is recommended.

If looping over all tables in a metastore is required, it is recommended to instead use system.information_schema.tables as your source instead of calling spark.sql(). For this case in particular, you can filter by table_type to filter out materialized views.