cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results forย 
Search instead forย 
Did you mean:ย 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results forย 
Search instead forย 
Did you mean:ย 

DLT maintenance job got stuck

tomnguyen_195
New Contributor III

โ€Ž06-30-2022 09:31 AM

Hi all,

Recently we just realize a huge cost associate with our databricks account and the main culprit of it is DLT's pipeline maintenance job that got auto-scheduled to run but got stucked and cost us thousand of DBU. Do you know what would be the reason for these jobs to get stuck randomly and how to avoid this huge cost?

Thanks,

Tom

Labels:
2 REPLIES 2

tinai_long
New Contributor III

โ€Ž06-30-2022 09:41 AM

Same question. These maintenance jobs run for the maximum timeout (168 hours) and do not terminate. Example below:

Screen Shot 2022-06-30 at 11.39.44 PM

Preview file
176 KB

jose_gonzalez
Moderator
Moderator
In response to tinai_long

โ€Ž08-29-2022 03:46 PM

do you see any issues? Could you share the full error stack trace?

0 Kudos
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!

Announcements