- 16056 Views
- 4 replies
- 3 kudos
Resolved! Using "Select Expr" and "Stack" to Unpivot PySpark DataFrame doesn't produce expected results
I am trying to unpivot a PySpark DataFrame, but I don't get the correct results.Sample dataset:# Prepare Data data = [("Spain", 101, 201, 301), \ ("Taiwan", 102, 202, 302), \ ("Italy", 103, 203, 303), \ ("China", 104, 204, 304...
- 16056 Views
- 4 replies
- 3 kudos
Latest Reply
- 3 kudos
You can also use backticks around the column names that would otherwise be recognised as numbers.from pyspark.sql import functions as F unpivotExpr = "stack(3, '2018', `2018`, '2019', `2019`, '2020', `2020`) as (Year, CPI)" unPivotDF = df.select("C...
- 3 kudos