cgrant
Databricks Employee
Databricks Employee

@vmpmreistad 

It is not possible to modify how listing is done. The standard recommendation is to switch from directory listing mode to file notification mode with large directories. If you have any control over the structure of your source data, I do not recommend having data at a minute/second granularity, especially on ADLS gen2.