How to get the files with a prefix in Pyspark from s3 bucket?

User16826994223 — Tue, 15 Jun 2021 16:22:18 GMT

I have different files in my s3. Now I want to get the files which starts with cop_

Re: How to get the files with a prefix in Pyspark from s3 bucket?

User16826994223 — Tue, 15 Jun 2021 16:23:05 GMT

You are referencing a FileInfo object when calling

.startswith()

and not a string.

The filename is a property of the

FileInfo

object, so this should work

filename.name.startswith('cop_ ')

should work.