Thanks!! Shamen Paris
The first two point has ruled out as the folder structure was not that complex and having very minimal data. Also, I did add the trailing slash in the query, it was a typo miss in my question.

I was confused why Iam able to browser all files from UI and not from the cluster.
So, third point do hint out on the cluster network, I checked the connectivity between the cluster network and PEP of storage account both are hosted in different Vnet. So, enabling the vnet Peering resolved the issue. Thankyou !!