Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
yesterday
Hi Vikas,
You may share the code directly in a message.
You can follow below
- Enable case-sensitive mode spark.conf.set("spark.sql.caseSensitive", "true") & validate the code
- Rename Fields immediately after parsing & validate the code.
parsed_trade_df = (
trade_raw_df
.select(from_json(col("value").cast("string"), trade_schema).alias("json"))
)
changed_df = parsed_trade_df
.select(col("json.t").alias("trade_id"),col("json.T").alias("trade_time_ms"), col("json.e").alias("event_type"))
)