Hi,
I have data pipeline which is running continuously, processes the micro batch data and store data in delta lake. This is taking care of any new data.
But at times, I need to process historical data without disturbing real time data processing.
Is there any suggested approach for this scenario. Appreciate any help.
Regards,
Sanjay
