jose_gonzalez
Databricks Employee
Databricks Employee

Hi @Alon Nisser​ ,

if you change this shuffle partitions configuration, the conf is persisted in the checkpoint, therefore the stream itself will continue to use the old value for stateful aggregations. If you want to use a new value, then you will need to use a new checkpoint.

View solution in original post