How to use the same job cluster in diferents job runs inside the one workflow

[Eiki]

I created a Workflow with notebooks and some job runs, but I would to use only one job cluster to run every job runs, without creating a new job cluster for each job run. Because I didn't want to increase the execution time with each new job cluster instantiated for each job run!

[brockb]

Hi,

If I understand correctly, you are hoping to reduce overall job execution time by reducing the Cloud Service Provider instance provisioning time. Is that correct?

If so, you may want to consider:

• Using a Pool of instances: https://docs.databricks.com/en/compute/pool-index.html
• Enable Serverless Compute for Workflows (currently in Public Preview at time of writing) https://docs.databricks.com/en/workflows/jobs/run-serverless-jobs.html 
• If the above considerations do not work for you, given that Jobs Clusters are designed to start when a job begins and stop when a job completes, you may consider creating a persistent All-Purpose cluster

Hope this helps.