basically you are looking for an event based trigger, where the event is the arrival of a new file.

It´s been a while since I worked on AWS but doesn´t Glue have a functionality like this?

On Azure I do exactly the same thing: when a file arrives in a certain location, a data pipeline starts containing a dbrx notebook.

If you want minimal latency, I suggest you use a pool with active workers, but that comes with a price ofc.