Hi @K_Anudeep @szymon_dybczak how do i understand a situation when 100 jobs are running in parallel with minimal latency needed. does autoloader directly connect to the cloud queue service ? or databricks stores and manages detected files somewhere? any azure cloud limit in this design?

Br