I have to process data for n number of devices which is sending data in every 5 seconds.

I have a similar scenario where I have to take last 3 hours of data and process it for all the devices for some key parameters. Now if I am doing it sequentially its taking a lot of time. I needed my job run should be completed within half an hour and save the process data in deltalake.

Can anyone please suggest a suitable approach to resolve this problem statement