I don't know enough about your configuration and use case to really answer this. There shouldn't be duplicates written by the pub/sub connector, as they are deduped in RocksDB. However they are deduped using the messageId field, which may be different to your "unique event ID". There is no way of the pub/sub connector deduping on something other than the messageId, so if there are dupliate records coming in to pub/sub, then you would stil need to use dropDuplicatesWithinWatermark to get rid of them.