unable to replace null with 0 in dataframe using Pyspark databricks notebook (community edition)

[db-avengers2rul]

Hello Experts,

I am unable to replace nulls with 0 in a dataframe ,please refer to the screen shot

from pyspark.sql.functions import col
emp_csv_df = emp_csv_df.na.fill(0).withColumn("Total_Sal",col('sal')+col('comm'))
display(emp_csv_df)

erorr

 

desired output

 

any suggestions ?

Regards,

Rakesh

[Hubert-Dudek · Accepted solution]

I bet that it is not real null but the string "null". Please check what is in the source and try luck with replacing it.

[Kaniz_Fatma]

Hi @Rakesh Reddy Gopidi​ ​, We haven’t heard from you since the last response from @Hubert Dudek​ , and I was checking back to see if you have a resolution yet.

If you have any solution, please share it with the community as it can be helpful to others. Otherwise, we will respond with more details and try to help.

Also, Please don't forget to click on the "Select As Best" button whenever the information provided helps resolve your question.