Loading spark dataframe to Mongo collection isn't allowing nulls
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-26-2024 07:14 PM
I am using below command to push DataFrame to Mongo Collection.
There are few null values in String and Double datatype columns , we see these are getting missed when pushed to mongo even after using the option("ignoreNullValues", false)
inputproddata.write.format("mongo").option("spark.mongodb.output.uri", connectionString).option("database",database).option("collection",collection).option("ignoreNullValues", false).mode("append").save()
Please let me know why this option does not work and suggest for any alternative solution
Labels:
- Labels:
-
Spark
1 REPLY 1
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-14-2024 05:11 AM
First approach works.. Thanks

