topic Re: Cannot find such info if Databricks supports nested jobs or tasks. For example, I have a 'job_a', which contains list of tasks, and another &#... in Data Engineering

Cannot find such info if Databricks supports nested jobs or tasks. For example, I have a 'job_a', which contains list of tasks, and another &#...

yzhang — Mon, 27 Mar 2023 23:04:49 GMT

Cannot find such info if Databricks supports nested jobs or tasks. For example, I have a 'job_a', which contains list of tasks, and another 'job_b', also contains a list of tasks. Now I'd like to have a 'job_all' that will run both 'job_a' and 'job_b', is it possible in Databricks?

Re: Cannot find such info if Databricks supports nested jobs or tasks. For example, I have a 'job_a', which contains list of tasks, and another &#...

LandanG — Tue, 28 Mar 2023 00:19:53 GMT

Hi @Yanan Zhang ,

We have a feature currently in private preview that will allow you to do this. As this is in private preview the best workaround until then would probably be to use the jobs API from a notebook and run them.

Hope that helps

Re: Cannot find such info if Databricks supports nested jobs or tasks. For example, I have a 'job_a', which contains list of tasks, and another &#...

Anonymous — Tue, 28 Mar 2023 04:08:45 GMT

Hi @Yanan Zhang

Thank you for posting your question in our community! We are happy to assist you.

To help us provide you with the most accurate information, could you please take a moment to review the response and select the one that best answers your question?

This will also help other community members who may have similar questions in the future. Thank you for your participation and let us know if you need any further assistance!

Re: Cannot find such info if Databricks supports nested jobs or tasks. For example, I have a 'job_a', which contains list of tasks, and another &#...

yzhang — Tue, 28 Mar 2023 14:12:57 GMT

Thank you, do you have roughly release time?

Re: Cannot find such info if Databricks supports nested jobs or tasks. For example, I have a 'job_a', which contains list of tasks, and another &#...

LandanG — Tue, 28 Mar 2023 15:02:02 GMT

@Yanan Zhang Nothing is set in stone, but I'd expect H1 this year for public preview (aka hopefully around summer). Depending on if you have a Databricks Rep or SA you could also ask them to inquire about the private preview

Re: Cannot find such info if Databricks supports nested jobs or tasks. For example, I have a 'job_a', which contains list of tasks, and another &#...

yzhang — Tue, 28 Mar 2023 15:19:43 GMT

Thank you! We will try the workaround now and wait for the public review to use the nest job/task function. I believe this is an important one otherwise hard to organize large set of jobs.