cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Workflow job tasks waits

Leszek1
New Contributor II

Hi,

I'm having issues with Workflow Pipelines since 3-4 days. The performance is degraded and very strange behavior of the Pipeline is that Tasks waits ~2-3 minutes to start executing code in the Notebook.

This is visible when you look at one of the tasks (no parallel tasks) --> time of running 4 cells is ~14s, time of running the task 2m 55s.

Does anyone have or had such issues?

Thanks

1 REPLY 1

Walter_C
Databricks Employee
Databricks Employee

Hello are you still behaving this issue? Are you counting the time it is taking for the cluster to start up or cluster was already running or using Serverless?

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group