Fixed interval micro-batches and AvailableNow Trigger

[Maatari]

What is the fundamental difference between Fixed interval micro-batches and AvailableNow Trigger, given that both can consume in micro-batch based on the desire size of the micro batch ? 

Is the fundamental difference the fact that AvailableNow shut down when finished and Fixed interval micro-batches never shut down ?

Indeed as far as i am understanding the documentation, AvailableNow does not mean, one micro-batch of everything available, but depending on the size set up, might mean consuming multiple micro-batch up to what was available when the job was triggered. Am I understanding this correctly ?

The other thing i find confusing in the documentation is the idea that the micro-batch size is set up by a property of type maxBytePerTrigger (depending on the data source). If AvailableNow represent one trigger then that is a problem. So does AvailableNow actually means multiple triggers ?

[Kaniz]

Hi @Maatari, 

• Fixed Interval Micro-batches are like clockwork, processing data at regular intervals without stopping.
• AvailableNow Trigger is more adaptive, consuming data as it becomes available and then gracefully shutting down.

Regarding your confusion about the micro-batch size:

• The maxBytesPerTrigger property determines the maximum size of each micro-batch.
• For AvailableNow, it doesn’t represent just one trigger; it can indeed involve multiple triggers if there’s more data to process.