03-08-2022 06:01 AM
Can I suggest customer about databricks Job orchestration (public preview) ? Can we use this feature in Production as well?
04-17-2022 10:58 AM
@ANOOP V , At present, we don’t have these features.
I assume we do plan to include it by Q4 FY2022.
What is next for Multitask Jobs?
After GA we will be working on some highly-requested features during the private preview:
- Job cluster reuse: make it possible to set a job cluster that can be shared across multiple tasks within the same job.
- Task run repair and retry: make it possible to manually restart a DAG execution from a failed step.
03-08-2022 07:15 AM
If something is in public preview you can use it in production, just things in preview are not covered by SLAs etc.
03-08-2022 09:12 AM
Thanks to @Hubert Dudek ! Any idea when the features like reuse a cluster across tasks in a job and restart the DAG (such that it only runs the tasks again that had previously failed). are available?
04-17-2022 10:58 AM
@ANOOP V , At present, we don’t have these features.
I assume we do plan to include it by Q4 FY2022.
What is next for Multitask Jobs?
After GA we will be working on some highly-requested features during the private preview:
- Job cluster reuse: make it possible to set a job cluster that can be shared across multiple tasks within the same job.
- Task run repair and retry: make it possible to manually restart a DAG execution from a failed step.
04-25-2022 04:03 AM
Hi @ANOOP V , I hope all your questions are answered by @Atanu Sarkar . Do you need some more help?
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
Click here to register and join today!
Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.