Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-07-2022 06:32 AM
ok so let us keep it simple:
df1 = spark.read.format("csv").option("header","true").load("file:///home/cloudera/data/a.csv")
if "col" in df1.columns:
df2 = df1
else:
df2 = df1.withColumn("col", lit(None).cast(TypeYouWant)Next you can use your case etc on df2.