@MoJaMa thanks for your response,

The reason I want to be on classic is the costs. Serverless runs have tripled our costs sadly. I have tried the query in case on pro and it will finish in two minutes. Cost indication running only on pro do not seem to fit the budget. Therefor the choice of serverless for now.

spark_conf dictionary you can set from within the model inside the config list and noticed those settings are being picked up. You can also set these in dbt_project.yml if you want to apply to a whole schema or just one model.

This explode you are seeing is not intentionally. Longer running but similar structured queries in the same schema have this as well. Those finish within a few minutes. Seems like databricks optimizing/restructuring the query results in this explode for some reason. There must be some databricks logic to it I guess. It is clear that pro and serverless are better at handling this.