nayan_wylde
Esteemed Contributor II

Here are few options  you can try and see if it resolves your issue.

1. SQL Warehouse Tuning

Use Serverless SQL Warehouse with Photon for faster spin-up and query execution. [docs.getdbt.com]
Size Appropriately: Start with Medium or Large, and enable auto-scaling for concurrency.
Keep Warehouse Warm: Schedule a lightweight query every 10–15 minutes to prevent cold starts.

2. Microbatch Optimization

Reduce Lookback: Try lowering from 48h to 24h or 12h, especially if late-arriving data is rare.
Set event_time on all upstream models to avoid full scans.
Tune concurrent_batches: Explicitly set this to a lower value (e.g., 2–4) to reduce parallel query load.

3. dbt Cloud Job Resiliency

Enable Job Retries: Configure retries with exponential backoff in dbt Cloud.
Split Models into Multiple Jobs: Break the 27 models into logical groups to reduce thread contention.
Use dbt Artifacts for Monitoring: Track model run times and failures using the dbt_artifacts package.