Save dataframe to the same variable
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-12-2023 12:45 AM
I would like to know if there is any difference if I save dataframe during tranformation to itself as first code or to new dataframe as second example.
Thanks
log_df = log_df.withColumn("process_timestamp",from_utc_timestamp(lit(current_timestamp()),"Europe/Amsterdam")) or
new_df = log_df.withColumn("process_timestamp",from_utc_timestamp(lit(current_timestamp()),"Europe/Amsterdam"))