Hello expert
I am new to spark. I am using same price of code but getting different results
When i am using below piece of code, i am getting error
py4j.Py4JException: Method or([class java.lang.String]) does not exist
df.filter(F.col("state").isNull()
| F.col("state")==""
| F.col("state").contains("")
| F.col("number").isNull()).show()
However when i am using below piece of code, its working fine
df.withColumn("state",
F.when(F.col("state")=="",None)
.otherwise(F.col("state"))).show()
The same F.col("state")=="" code is working in one place but not working in other