@NandiniN thanks for you response,

I tried influencing the way skewjoin is being applied and the way files are written but no results.

This query is part of a businessvault run and is called from dbt. It is not a job run native to databricks and sadly I cannot simulate this run from a job with a dbt command as a classic wh is unavailable to use.