Plot number of abandoned cart items by product

SSV_dataeng
New Contributor II
abandoned_carts_df = (email_carts_df.filter(col('converted') == False).filter(col('cart').isNotNull()))
display(abandoned_carts_df)
 SSV_dataeng_0-1690194232666.png
abandoned_items_df = (abandoned_carts_df.select(col("cart").alias("items")).groupBy("items").count())
display(abandoned_items_df)

verfiication Failed

expected_columns = ["items", "count"]
expected_count = 12
assert abandoned_items_df.count() == expected_count, "Counts do not match"
assert abandoned_items_df.columns == expected_columns, "Columns do not match"
print("All test pass")
 
AssertionError: Counts do not match
--------------------------------------------------------------------------- AssertionError Traceback (most recent call last) <command-1787238488083704> in <cell line: 5>() 3 expected_count = 12 4 ----> 5 assert abandoned_items_df.count() == expected_count, "Counts do not match" 6 7 assert abandoned_items_df.columns == expected_columns, "Columns do not match" AssertionError: Counts do not match

daniel_sahal
Databricks MVP

@SSV_dataeng 
What is the question here?

NandiniN
Databricks Employee
Databricks Employee

Hi @SSV_dataeng ,

Try 

abandoned_items_df = (abandoned_carts_df
.withColumn("items", explode("cart"))
.groupBy("items")
.count()
.sort("items")
)