Differences between lit(None) or lit(None).cast('string')

SaraCorralLou
New Contributor III

I want to define a column with null values in my dataframe using pyspark. This column will later be used for other calculations.

What is the difference between creating it in these two different ways?

  • df.withColumn("New_Column", lit(None))
  • df.withColumn("New_Column", lit(None).cast('string'))

Can both be used? Is there a wrong one?

Thank you so much!