Catch when a notebook fails and terminate command in threaded parallel notebook run

[cmilligan]

I have a command that is running notebooks in parallel using threading. I want the command to fail whenever one of the notebooks that is running fails. Right now it is just continuing to run the command.

Below is the command line that I'm currently running:

q = Queue()
worker_count = 3 
 
def run_notebook(notebook):
  print(notebook)
  dbutils.notebook.run(notebook, 30, {"begin_date": begin_date, "end_date": end_date})
 
def run_tasks(function, q):
    while not q.empty():
      value = q.get()
      function(value)
      q.task_done()
 
for x in range(len(notebook_steps)): 
  for x in notebook_steps[x]:
    q.put(x)
      
  for i in range(worker_count):
    t=Thread(target=run_tasks, args=(run_notebook, q))
    t.daemon = False
    t.start()
 
  q.join()