Autoloader - Ingestion of CSV files when there is not operation column

[2vinodhkumar]

Hi,

We are working on ingesting multiple files from S3. The files name are fixed based on our source system, Files get replaced frequently with full feed. In DLT when we process new file we have to delete the records processed earlier of the same file name. As part of documentation suggestion apply_changes was the option given to us, but as we process same file multiple times and as we have hard deleted records from file. We are unable to do apply_changes efficiently for deleted records.

Regards,

Vinodh Kumar K

[2vinodhkumar]

@Retired_mod 

I have seen these solutions from previous question. Issue is apply_changes is not working when record is hard deleted from files.

Also a suggestion for this community portal UI, the captcha verification is taking too much time. Like verifying bicycle,traffic lights..etc. On checking correct captcha it's not able to proceed, it asks for more than 20 verifications