dbutils.fs.ls doesn't work on Unity catalog managed location

[venkad]

Hi,

dbutils.fs.ls() is a most useful utility to list the files in remote cloud storage. I cannot use this function on top of the location managed by the Unity catalog.

I have a managed external location created on `'abfss://xyz@abcd.dfs.core.windows.net/` where my schema is created. When i try to list the files created by the tables inside this schema, i get this error.

```

%py display(dbutils.fs.ls('abfss://xyz@abcd.dfs.core.windows.net/__unitystorage/schemas/5551e5b7-dd3f-45cb-94ce-a572e3ff7760/tables/0fbd721d-1701-4516-ad09-d04a3057e016'))

```

Error message

```

AnalysisException: [RequestId=80fd99cc-efd3-44c3-86ae-17bbd5e699bb ErrorClass=INVALID_PARAMETER_VALUE.LOCATION_OVERLAP] Input path url 'abfss://xyz@abcd.dfs.core.windows.net/__unitystorage/schemas/5551e5b7-dd3f-45cb-94ce-a572e3ff7760/tables/0fbd721d-1701-4516-ad09-d04a3057e016' overlaps with managed storage

```

I tried to run **list** command as well. It works only on the exact location. But throws error when tries to list the directory inside it.

Below code works.

```

list 'abfss://xyz@abcd.dfs.core.windows.net/'

```

Below code doesn't work.

```

list 'abfss://xyz@abcd.dfs.core.windows.net/__unitystorage/'.

```

Is this an expected behavior? If yes, we are missing a great utility in the unity catalog.

Is there any work around or am I missing something?

[Kaniz]

Hi @Venkadeshwaran K​, The error message you received is an analysis exception that indicates an issue with the input path URL you provided. Specifically, the error message suggests that the input path URL overlaps with managed storage.

To resolve this issue, you may need to adjust the path URL you are using to access the data. You could try changing the course URL. Additionally, you may want to check the permissions and access controls on the data you are attempting to access to ensure that you have the necessary privileges to access it.

[Anonymous]

This is an expected behaviour. You trying to query the managed location of a catalog and this path is to be accessed by UC only, so abfss://xyz@abcd.dfs.core.windows.net/__unitystorage/* won't be accessible. Any subdirectories within xyz container will work.

list 'abfss://xyz@abcd.dfs.core.windows.net/<sub_directory> should work

[Anonymous]

Hi @Venkadeshwaran K​ 

Thank you for posting your question in our community! We are happy to assist you.

To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?

This will also help other community members who may have similar questions in the future. Thank you for your participation and let us know if you need any further assistance!