szymon_dybczak
Esteemed Contributor III

Hi @MyTrh ,

Ok, I think I created similiar use case to yours. I have streaming table with column structure based on your example

image.png

CREATE OR REFRESH STREAMING TABLE clicks_raw AS SELECT *, current_timestamp() as load_time FROM cloud_files('/Volumes/dev/demo_db/landing_zone/clicks_data', "csv", map("cloudFiles.inferColumnTypes", "true"))

 

Now, I have another streaming table which is consuming the first one. Here I perform grouping to achieve distinct combination of user_id and department_id and I'm aggregating count for each group.

So as you can see from sceen below, take for example user_id = user_001 and departament_id = dept_01.

We can see that in clicks_raw there are two rows for this combination:

 

        - one with 20 clicks

        - another one with 15 clicks

 

So the final result in our target table for this user and this department should be 35. 

 

image (1).png

 

CREATE OR REFRESH STREAMING TABLE clicks_aggregated AS SELECT user_id, department_id, SUM(clicks) as clicks FROM STREAM(LIVE.clicks_raw) GROUP BY user_id, department_id

 As you can see from screen below I achieved incremental refresh.

image (2).png

 

 

 

 

 

 

 

View solution in original post