cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

ckwan48
by New Contributor III
  • 1876 Views
  • 4 replies
  • 4 kudos

Date schema issues with pyspark dataframe creation

I'm having some issues with creating a dataframe with a date column. Could I know what is wrong?from pyspark.sql import SparkSession from pyspark.sql.types import StructType from pyspark.sql.types import DateType, FloatType spark = SparkSession.bui...

  • 1876 Views
  • 4 replies
  • 4 kudos
Latest Reply
ckwan48
New Contributor III
  • 4 kudos

Hi @Kaniz Fatma​,I actually changed the date format to 'M/d/Y' and it didn't throw any errors. I found in my csv file that it had dates like '3/1/2022'. Could that be the issue? But some dates also were like '12/1/2022. So I'm kind of confused.

  • 4 kudos
3 More Replies
Labels