- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-26-2019 05:29 AM
Hi,
It is strange that it returns null. It works fine for me in pyspark as well. Could you please compare the code? Also try displaying the earlier dataframe. pls make sure that the values in original dataframe are displaying properly and are in appropriate datatypes (StringType).
```
from pyspark.sql.functions import unix_timestamp, colfrom pyspark.sql.types import TimestampType
from pyspark.sql.types import StringType
df = spark.createDataFrame(["2015/01/01 03:00:36"], StringType()).toDF("ts_string")
df1 = df.select(unix_timestamp(df.ts_string, 'yyyy/MM/dd HH:mm:ss').cast(TimestampType()).alias("timestamp"))
df1.show()
```
If it still doesn't resolve, please share the full code, including how you are creating the original dataframe. Please let us know how it goes.
Thanks