Error of "name 'IntegerType' is not defined" in attempting to convert a DF column to IntegerType

[BingQian]

initialDF .withColumn("OriginalCol", initialDF.OriginalCol.cast(IntegerType))

Or

initialDF .withColumn("OriginalCol", initialDF.OriginalCol.cast(IntegerType()))

However, always failed with this error :

NameError: name 'IntegerType' is not defined

Any help is appreciated.

[KristoRaun · Accepted solution]

I'm assuming you are using Python.

In order to use the IntegerType, you first have to import it with the following statement:

from pyspark.sql.types import IntegerType

If you plan to have various conversions, it will make sense to import all types. This can be done as follows:

from pyspark.sql.types import *

For further reference, you can check the Spark documentation:

https://spark.apache.org/docs/latest/sql-reference.html

[BingQian]

Thank you @Kristo Raun​  !