Cannot Convert Column to Bool Error - When Converting dataframe column which is in string to date type in python

desai_n_3
New Contributor II

Hi All,

I am trying to convert a dataframe column which is in the format of string to date type format yyyy-MM-DD?

I have written a sql query and stored it in dataframe.

df3 = sqlContext.sql(sqlString2)
df3.withColumn(df3['CalDay'],pd.to_datetime(df3['CalDay'],format = '%Y%m%d'))

I get this error

ValueError: Cannot convert column into bool: please use '&' for 'and', '|' for 'or', '~' for 'not' when building DataFrame boolean expressions.

How is this happening?