Tharun-Kumar
Databricks Employee
Databricks Employee

@smurug 

Job Cluster has been designed to be unique for each run of a job. So, each run of your job would run against a new job cluster.

If you want your job to run continuously without any delay and to re-use the cluster, I would recommend to use a dedicated interactive cluster. In this case, the cluster would be retained across job runs and your job runs would be instantly executed after the previous run is completed.