- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-23-2020 10:00 PM
initialDF .withColumn("OriginalCol", initialDF.OriginalCol.cast(IntegerType))
Or
initialDF .withColumn("OriginalCol", initialDF.OriginalCol.cast(IntegerType()))
However, always failed with this error :
NameError: name 'IntegerType' is not defined
Any help is appreciated.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-24-2020 02:03 PM
I'm assuming you are using Python.
In order to use the IntegerType, you first have to import it with the following statement:
from pyspark.sql.types import IntegerType
If you plan to have various conversions, it will make sense to import all types. This can be done as follows:
from pyspark.sql.types import *
For further reference, you can check the Spark documentation:
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-24-2020 06:50 PM
Thank you @Kristo Raun !