- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-30-2025 07:48 AM
Here are few options you can try and see if it resolves your issue.
1. SQL Warehouse Tuning
Use Serverless SQL Warehouse with Photon for faster spin-up and query execution. [docs.getdbt.com]
Size Appropriately: Start with Medium or Large, and enable auto-scaling for concurrency.
Keep Warehouse Warm: Schedule a lightweight query every 10–15 minutes to prevent cold starts.
2. Microbatch Optimization
Reduce Lookback: Try lowering from 48h to 24h or 12h, especially if late-arriving data is rare.
Set event_time on all upstream models to avoid full scans.
Tune concurrent_batches: Explicitly set this to a lower value (e.g., 2–4) to reduce parallel query load.
3. dbt Cloud Job Resiliency
Enable Job Retries: Configure retries with exponential backoff in dbt Cloud.
Split Models into Multiple Jobs: Break the 27 models into logical groups to reduce thread contention.
Use dbt Artifacts for Monitoring: Track model run times and failures using the dbt_artifacts package.