I have a stateless streaming application that uses foreachBatch. This function executes between 10-400 times each hour based on custom logic. The logic within foreachBatch includes: collect() on very small DataFrames (a few megabytes) --> driver mem...
