Python library not installed when compute is resized

Bilel
New Contributor II

 

Hi,

I have a python notebook workflow that uses a job cluster. The cluster lost at least a node (due to Spot Instance Termination) and did an upsize. After that I got an error in my job "Module not found", but the python module was being used before the cluster lost the node. So I guess the library was not installed in the new node. This is the first time it happens in this workflow when a node restarts. The cluster I use has 4 workers.

Any idea what might be going wrong? Thanks !