I'm curious about the difference between using S3's SQS to set the queue url in spark's readStream option and AutoLoader reading from Cloudfiles.

I would also like some advice on which one is better to use in which situation.
(from a cost and performance perspective)

I'm planning to create data in real-time using complex operations such as JOIN on the data imported from real-time ingestion.