We have Data bricks Jobs that run a combination of DLT pipelines and notebook tasks.
For the notebook tasks, we use a job cluster, and we are able to auto‑terminate it after execution using auto termination: 10
However, the DLT-managed clusters behave differently — after the pipeline completes, the DLT cluster continues running for up to 60 minutes before shutting down. This results in unnecessary additional cost.
Is there any way to reduce the idle time or auto‑terminate DLT-managed clusters sooner, similar to job clusters? Or any configuration available to control the DLT cluster shutdown time?
Any guidance or recommended best practices would be appreciated.
