DBT task status update gets delayed for several minutes

[Rosty]

Hi, our team has recently begun experiencing a several-minute delay between Databricks DBT tasks finishing the computations and the subsequent status update from running state to success.  The DBT project is part of the workspace git repo. In the first screenshot, you can see the triggered task logs suggesting the processing of the dbt model has finished with OK status (duration is 1m 21s), but the task status remains 'Running' for another several minutes (and usually gets updated to 'Success' at ~6 mins duration time). As a result, the tasks that previously took 1-2 minutes to execute from Airflow now take 6-7, adding a significant delay to the overall pipeline execution. The DBT task orchestration approach has not been updated for a long time, with the same all-purpose compute and SQL warehouses being used. Similar behaviour was observed in both development and qa environments with different compute clusters. Would be grateful for any ideas on the potential root causes or suggestions for troubleshooting steps, thanks!

[Brahmareddy]

Hi Rosty,

How are you doing today? thanks for sharing the detailed context. I agree, it definitely sounds frustrating to have DBT tasks showing delays even after finishing the actual work. Based on what you've described, the delay is likely happening after computation, during the finalization or cleanup phase, where Databricks updates task metadata, writes logs, or commits final status to the control plane. Since this delay happens consistently across environments and without recent orchestration changes, it might be related to background processes like job metadata syncing, Git integration handling, or even SQL warehouse scaling behavior. A few things you could try: enable detailed logging to check if Git operations or metadata operations are hanging; check workspace audit logs to see when task status updates are written; and consider testing with a job cluster (instead of all-purpose) or toggling Git repo integration temporarily to isolate the bottleneck. If the delay persists, it’s a good idea to open a support ticket with Databricks since they can dig into backend timing more deeply.

Regards,

Brahma