@Data_Mavericks, I have actually tried using your suggested way, but the table is empty after creating it. Not sure why.

A recently Databricks feature which can federate AWS Glue may help me out since we have these files in s3 defined in Glue as tables already.