I believe that is true as well, since I am just using the python library implementation of it. If there would be a way to run it sequentially on the driver but once one row has been processed it hands it off to the executors again and materialises it outside of memory I would be fine with it (my main problem lies in memory usage, not speed/cpu utilisation), but I don't know if there is a way to do this.

I'll have a look at OpenCV, though last time I checked I couldn't get the decryption to work in there.

Deepstream seems a completely end-to-end system and since I have some bespoke preprocessing to do I don't think I can make that work (easily).

Thanks for your answer!