Hi @Phani1 ,

Structure of folders that you are going to use make sense to me. Since you've mentioned that there will be thousands of files, the best practice will be to use autoloader with file notification mode. 

Also, you can read about databricks recommendations: 

https://learn.microsoft.com/en-us/azure/databricks/ingestion/cloud-object-storage/auto-loader/file-n...

https://docs.databricks.com/en/ingestion/cloud-object-storage/auto-loader/production.html