09-20-2018 12:33 AM
09-20-2018 08:58 AM
You can use the .drop() function, which is available as part of DataFrameNaFunctions.
With Scala:
https://spark.apache.org/docs/2.1.0/api/java/org/apache/spark/sql/DataFrameNaFunctions.html#drop()
With Python:
09-20-2018 11:47 AM
@dillon,
Drop will not help me in my case , i have a 200 rows in a table which has 100 rows as null and rest of rows has proper values .
09-15-2019 07:33 AM
Simply you can just use the .drop() function, which is available as part of DataFrameNaFunctions.
09-26-2019 03:31 AM
@searro , it would be nice to have a well-formed example.
12-09-2019 04:10 AM
How can we do this in Java.
I have a DataFrame with three columns, When all three values are null. Is there a way to delete that row in java?
03-18-2020 10:44 AM
you can try this: df.na.drop(how = "all"), this will remove the row only if all the rows are null or NaN
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
Click here to register and join today!
Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.