Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-14-2022 02:41 AM
You actually don't need to filter `.csv` files like that.
You can use `pathGlobFilter` to do a regex match for selecting files that matches provided regular expression.
df = spark.read.option("pathGlobFilter","*.csv").csv(upload_path)