This is really interesting question. I bet that better is to use pool for servers for that jobs tasks so we can see in pool how servers behave and also have settings for that. From my experience VM exists for at least few minutes after job failed (when pool have min idle 0). What technically is going there after job failed in that case should rather be answered by someone from inside of databricks @Kaniz Fatma​