- 1265 Views
- 2 replies
- 0 kudos
KinesisSource generates empty microbatches when there is no new data.
Is it normal for KinesisSource to generate empty microbatches when there is no new data in Kinesis? Batch 1 finished as there were records in kinesis and BatchId 2 started. BatchId 2 was running but then BatchId 3 started . Even though there was no m...
- 1265 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Pranathi Girish​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answe...
- 0 kudos
- 840 Views
- 2 replies
- 0 kudos
How to run sparkStream for earlier (not future messages)
Hi, I'm listening to a stream for kinesis, don't need the data in real-time, so I could run it on an hourly basis looking to achieve two things:-Save money by don't have a cluster up 24/7-Have bigger files saved for each readThe stream is constant so...
- 840 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Alejandro Martinez​ , Here is a somewhat similar issue on S.O. Please look and let us know if that helped.
- 0 kudos