cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Databricks Workflow duration issues

ChsAIkrishna
New Contributor II

There is a discrepancy in the Databricks Workflow . Currently, the start and end times printed are when the workflow is triggered, not when the actual workflow execution begins. Ideally, it should log the task execution time. The issue is that the current implementation includes queuing time, which is not the intended behavior.

For example, if the threshold is set to 60 minutes, the expectation is that the task execution time should not exceed 60 minutes, but it is currently counting the time spent in the queue, which is not correct.

Any advise to fix ?

2 REPLIES 2

Walter_C
Databricks Employee
Databricks Employee

This is already reported on a feature request to make sure we set a difference in between the queue time and the execution time in the job UI page, this can be tracked with id DB-I-8963

Thank you for your reply and tracking ID

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group