If --py-files doesn’t work, try this shorter method:Create a Conda Environment: Install your packages.conda create -n myenv python=3.xconda activate myenvpip install your-packagePackage and Submit: Use conda-pack and spark-submit with --archives.cond...
Use --py-files with Spark Submit: Zip the package and add it using --py-files when you run spark-submit. For example:spark-submit --py-files path/to/your_package.zip your_script.py
