if they are already written, you have to bite the apple and read them (with spark/databricks or ADF data flow). Then you can coalesce/repartition them and write the merged files back to the data lake.

Another option is by using delta lake, using MERGE statement (incoming data is merged in the existing).

Delta lake handles the partitioning. But still: you have to initially read all those small files first and write them to delta lake.