dropDuplicates inside foreachBatch

MikeGo
Valued Contributor

Hi,

If I use dropDuplicates inside foreachBatch, the dropDuplicates will become stateless and no state. It just drop duplicates for the current micro batch so I don't have to specify watermark. Is this true?

Thanks