I got the answer from the Databricks Support on this.

The point which has been mentioned in doc "Once a table is created in a path, users can no longer directly access the files in that path even if they have been given privileges on an external location or storage credential to do so. This is to ensure that users cannot circumvent access controls applied to tables by reading files from your cloud tenant directly." is a bit different. Consider there is User U4 who is having access to the external location but DO NOT have access to the table T1. In such a scenario the aforementioned point is applicable and we raise an error like “PERMISSION_DENIED: trying to access path with conflicting external tables”.