Hello guys, i have some materialized views created in my databricks workspace and after 1 change in one of them, it became 3x slower (9 minutes to 30 minutes). After some debugging i found that the bottleneck process in the execution plan is one called "Columnar To Row" (108h in agg task time). For some reason, if i run the exact same query in a CREATE TABLE in serveless notebook, the execution plan do not have this "Columnar To Row" and it runs in 9 minutes. I know that MATERIALIZED VIEW runs in DLT serveless pipelines, it has some different optimizations than run in serveless notebooks? It is a really strange behavior.
