Tharun-Kumar
Databricks Employee
Databricks Employee

Hi @Gilg 

Is it ingesting the same number of files as before?

Also, you could try using Auto Loader with file notification mode. If there are too many files in the source directory, then significant amount of time would be spent on listing of the directory. We can validate this by analyzing the logs.