https://community.databricks.com/t5/data-engineering/how-are-parallel-and-subsequent-jobs-handled-by-cluster/m-p/11405#M6388 <P>Hello,</P><P></P><P>Apologize for dumb question but i'm new to Databricks and need clarification on following.</P><P>Are parallel and subsequent jobs able to reuse the same compute resources to keep time and cost overhead as low as possible vs. are they spinning a new cluster all the time?</P><P></P><P>Regards,</P><P>Tanja</P> Wed, 18 Jan 2023 11:59:03 GMT 138999 2023-01-18T11:59:03Z https://community.databricks.com/t5/data-engineering/how-are-parallel-and-subsequent-jobs-handled-by-cluster/m-p/11405#M6388 <P>Hello,</P><P></P><P>Apologize for dumb question but i'm new to Databricks and need clarification on following.</P><P>Are parallel and subsequent jobs able to reuse the same compute resources to keep time and cost overhead as low as possible vs. are they spinning a new cluster all the time?</P><P></P><P>Regards,</P><P>Tanja</P> Wed, 18 Jan 2023 11:59:03 GMT https://community.databricks.com/t5/data-engineering/how-are-parallel-and-subsequent-jobs-handled-by-cluster/m-p/11405#M6388 138999 2023-01-18T11:59:03Z https://community.databricks.com/t5/data-engineering/how-are-parallel-and-subsequent-jobs-handled-by-cluster/m-p/11406#M6389 <P>@tanja.savic tanja.savic​&nbsp;</P><P>You can use <B>shared job cluster</B>:</P><P><A href="https://docs.databricks.com/workflows/jobs/jobs.html#use-shared-job-clusters" alt="https://docs.databricks.com/workflows/jobs/jobs.html#use-shared-job-clusters" target="_blank">https://docs.databricks.com/workflows/jobs/jobs.html#use-shared-job-clusters</A></P><P></P><P>But remember that a shared job cluster is scoped to a single job run, and cannot be used by other jobs or runs of the same job. It means that 1 job = 1 cluster</P><P></P> Wed, 18 Jan 2023 12:55:02 GMT https://community.databricks.com/t5/data-engineering/how-are-parallel-and-subsequent-jobs-handled-by-cluster/m-p/11406#M6389 daniel_sahal 2023-01-18T12:55:02Z