external volume: overlap error with empty catalog?

[-werners-]

Hi all,

I try to create an external volume in a catalog/schema on a subpath of the external location.
This works fine, however for a certain path I cannot create an external volume.  I always get
Input path url overlaps with managed storage within 'CreateVolume' call, for this path only.

However, there is nothing in our catalogs (fresh dev environment). So there cannot be any overlap!
This being a dev environment, we messed around trying unity out, so we are thinking perhaps unity caches some metadata of removed catalogs/tables/...?
Is there any way to find out where the issue could be located?
I already looked in system.information_schema, but there is nothing to be found.

Any thoughts?

[Kaniz]

Hi @-werners-, The error message you're seeing suggests that the path you're using for your external volume overlaps with an approach used for managed storage. This can occur if the way you're using for your external book is a subpath of a path used for managed storage. 

Here are a few things you can try to resolve this issue:

1. **Check the path**: Make sure the path you're using for your external volume doesn't overlap with any paths used for managed storage. Even if you're working in a fresh dev environment, it's possible that some managed storage paths were set up previously and are causing this conflict.

2. **Check your permissions**: Ensure you have the necessary permissions to create an external volume. According to the Databricks documentation, you need the CREATE EXTERNAL VOLUME permission for the external location, USE SCHEMA and CREATE VOLUME for the schema, and USE CATALOG for the catalog.

3. **Clear cache**: If you've been experimenting with Unity, it's possible that some metadata is being cached and causing this issue. Try clearing your cache to see if this resolves the issue.

If these suggestions don't help, I recommend reaching out to Databricks support for further assistance by filing a support ticket.

[-werners-]

point 1 and 2 can't be it as I have full permissions on all and the catalog is empty.  Creating a volume on a subpath of this problematic directory works ...
How do I clear this cache?  You mean browser cache or workspace cache?

[-werners-]

I created a support request as this should not occur.

[Kaniz]

Our support team will look into the matter to determine the root cause and take appropriate action to prevent such occurrences in the future.

[-werners- · Accepted solution]

The issue disappeared by itself after a while (around 30 days) without Databricks support changing anything.
We (myself and databricks) suspect that a garbage collect or something freed locked paths.

[owlleg6]

I have the same issue. I have created a Catalog using External Location, deleted it and now i can't create Volume for this Location even thought Catalog is deleted long ago

[-werners-]

it is already a few months ago but I created a ticket for this. So the issue resolved itself after a while.
We believe that after 30 days Unity does a cleanup and the deleted items are scrapped completely.
Not sure if that is the case for you though.  If not, I'd create a ticket too (after checking there is not actual overlap of course).