Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-13-2017 07:42 AM
Actually, the intersection transformation does deduplication. If you don't need it, you can just slightly modify your code:
val filteredRdd = rddAll.filter(line => line.contains(token))
and send data of the rdd to your program by calling of an action like collect or take:
val result = filteredRdd.take(100)
The result is just regular Array.