Muliple Where condition vs AND && in Pyspark
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-09-2021 03:03 AM
.where((col('state')==state) & (col('month')>startmonth)
I can do the where conditions both ways. I think the one below add readability. Is there any other difference and which is the best?
.where(col('state')==state)
.where(col('month')>startmonth)
Labels:
1 REPLY 1
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-09-2021 03:04 AM
You can use explain to see what type of physical and logical plans are getting created . This is the best way to see difference , but as mentioned in the question , it should give the same physical plan

