- 1899 Views
- 2 replies
- 0 kudos
how to compute difference over time of a spark structure streaming?
I have a table with a timestamp column (t) and a list of columns for which I would like to compute the difference over time (v), by some key(k): v_diff(t) = v(t)-v(t-1) for each k independently.Normally I would write:lag_window = Window.partitionBy(C...
- 1899 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
I found this but could not make it work https://www.databricks.com/blog/2022/10/18/python-arbitrary-stateful-processing-structured-streaming.html
- 0 kudos