cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
cancel
Showing results for 
Search instead for 
Did you mean: 

Muliple Where condition vs AND && in Pyspark

User16826994223
Honored Contributor III

.where((col('state')==state) & (col('month')>startmonth)

I can do the where conditions both ways. I think the one below add readability. Is there any other difference and which is the best?

.where(col('state')==state)

.where(col('month')>startmonth)

1 REPLY 1

User16826994223
Honored Contributor III

You can use explain to see what type of physical and logical plans are getting created . This is the best way to see difference , but as mentioned in the question , it should give the same physical plan

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.