Databricks Community

tusworten · 02-01-2022

I'm begginner working with Spark SQL in Java API. I have a dataset with duplicate clients grouped by ENTITY and DOCUMENT_ID like this:.withColumn( "ROWNUMBER", row_number().over(Window.partitionBy("ENTITY", "ENTITY_DOC").orderBy("ID")))I added a ROWN...

tusworten · 02-07-2022

Hi @Kaniz Fatma Her answer didn't solve my problem but it was useful to learn more about UDFS, which I did not know.

Databricks Community

User Stats

User Activity

Spark SQL Group by duplicates, collect_list in array of structs and evaluate rows in each group.

Re: Spark SQL Group by duplicates, collect_list in array of structs and evaluate rows in each group.