Brahmareddy
Esteemed Contributor II

Spark's error happens because the worker nodes can't access your local files. Instead of using Spark to decrypt, try doing it outside of Spark using Python's multiprocessing or a simple batch script for parallel processing. Another option is to move the files to something like HDFS or S3 so all nodes can access them. Spark works best for processing data, not handling files directly.

View solution in original post