Hi,

i'm using concat_ws in scala to calculate a checksum for the dataframe, i.e.:

df.withColumn("CHECKSUM", sha2(functions.concat_ws("", dataframe.columns.map(col): _*), 512))

I have one example here with just 24 columns that already throws the following exception: org.apache.spark.sql.AnalysisException: cannot resolve 'concat_ws('', <list of the columns)

Any ideas what's happening? I assume the list get's too long (char wise), but I have no idea how to make this work.

Thanks!