Hubert-Dudek
Databricks MVP

Please use localCheckpoint(True) so it will be stored on executors and trigger immediately.


My blog: https://databrickster.medium.com/