I need to read/query table A, manipulate/modify the data and insert the new data into Table A again.
I considered using :
Cur_Actual = spark.sql("Select * from Table A")
currAct_Rows = Cur_Actual.rdd.collect()
for row in currAct_Rows:
do_somthing(row)
But that doesn't allow me to change the data, for example:
row.DATE = date_add(row.DATE, 1)
And then I don't understand how I would insert the new data into TABLE A.
Andy advice would be appreciated.