K_Anudeep
Databricks Employee
Databricks Employee

Hello @saurabh18cs !

You don’t need to choose queues and simply use the “File events” path. When enabled, Databricks uses one managed queue per external location (Unity Catalogue), and all your streams that read from that location share it. This avoids cloud queue limits, and Databricks auto-tunes the notification plumbing.

Dochttps://docs.databricks.com/aws/en/ingestion/cloud-object-storage/auto-loader/file-notification-mode...

If you stay in the current file notification mode, the default pattern is one queue per stream, and Databricks recommends fan-out only when you’re constrained by provider limits

I think throughput has no fixed per-second cap. By default, it processes up to 1,000 files per micro-batch; tune with cloudFiles.maxFilesPerTrigger

Please let me know if you have any further questions.

Anudeep