Databricks Community

supriya · 01-22-2016

I need to create new column with data in dataframe. Example:val test = sqlContext.createDataFrame(Seq( (4L, "spark i j k"), (5L, "l m n"), (6L, "mapreduce spark"), (7L, "apache hadoop"), (11L, "a b c d e spark"), (12L, "b d"), (13L, "spark f g h"), ...

supriya · 01-09-2017

@jack AKA karthik: For adding a timestamp in dataframe dynamically: import org.apache.spark.sql.functions._ val AppendDF = customerDF.withColumn("new_column_name",current_timestamp()) I think it's work for you.

supriya · 01-31-2016

Thanx @Raela Wang . But my requirement is different, i want to add Average column in test dataframe behalf of id column. I know this one is possible using join ...but I think join process is too slow. If you have any other solution then you can sug...

Databricks Community

User Stats

User Activity

How to append new column values in dataframe behalf of unique id's

Re: How to append new column values in dataframe behalf of unique id's

Re: How to append new column values in dataframe behalf of unique id's