duplicate data published in kafka offset

dipali_globant
Databricks Partner

we have 25k data which are publishing by batch of 5k.

we are numbering the records based on row_number window function and creating batch using this.

we have observed that some records like 10-20 records are getting published duplicated in 2 offset. 

can someone help with the probable cause for this issue.