I have a notebook where I bring info about a previous task run metadata from the API ".... /jobs/runs/get". The response should be a dictionary that contains information such as task key, run if, run page URL etc.
When I run the notebook as part of a scheduled ETL workflow the notebook failed with a key error that the 'result state' key was not found in the 'state' dictionary. However when I run it manually, the notebook didn't fail and the 'result state' key contained a valid value.
How is it possible to have different result between manual and automated task run?
Is there any case that the state of the job or tasks might have been different when the notebook was executed automatically compared to when I ran it manually. For example, the job might have completed and populated all the expected keys by the time I ran it manually.
