Lakeflow jobs file trigger thru overwritten files

[DynDe]

Is there any way a job can be triggered with overwritten files through a file trigger?

I knew the limitations, but I just want to have other opinions or solutions.

Limitations

• Only new files trigger runs. Overwriting an existing file with a file of the same name does not trigger a run.

Thank you in advance!

[Ashwin_DSA]

Hi @DynDe,

Unfortunately, you can't do that as of today.

Is there a specific reason you want to maintain the same name for the triggers? Instead of linking to a static file name, could you point to a root path so that any new files will trigger the job? If this approach is acceptable, you might consider appending a value, like a timestamp, to the files each day, which would allow them to be recognised as new files?

The other option is to schedule it to run every few minutes or hours. This approach should work, but it’s not event-driven, to be honest.

If this answer resolves your question, could you mark it as “Accept as Solution”? That helps other users quickly find the correct fix.