maxFilesPerTrigger: This option specifies how many new files should be considered in every micro-batch. The default value is 1000.
maxBytesPerTrigger: This option sets a soft maximum on the amount of data processed in each micro-batch. It is not set by default but can be configured to limit the data processed per batch.
