sorry I misunderstood.

The reason you see this many parquet files is because parquet is immutable, you cannot change it (and parquet is used by delta lake). So if a new event/record is sent, the stream will process it and append it to the table, which means a new file too.

If you continuously write data to a Delta table, it will over time accumulate a large number of files, especially if you add data in small batches. This can have an adverse effect on the efficiency of table reads, and it can also affect the performance of your file system. Ideally, a large number of small files should be rewritten into a smaller number of larger files on a regular basis. This is known as compaction.

https://docs.delta.io/latest/best-practices.html#compact-files

Another option is to read the events less often, and go for a more batch-based approach.