- 1434 Views
- 2 replies
- 2 kudos
Pyspark will error while I pack source zip package without dir.
If I send the package made by zipfile on spark.submit.pyFiles which zip by this code. import zipfile, os def make_zip(source_dir, output_filename): with zipfile.ZipFile(output_filename, 'w') as zipf: pre_len = len(os.path....
- 1434 Views
- 2 replies
- 2 kudos
Latest Reply
- 2 kudos
I checked, and your code is ok. If you set source_dir and output_filename please remember to start path with /dbfsIf you work on the community edition you can get problems with access to underlying filesystem.
- 2 kudos