cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results for 
Search instead for 
Did you mean: 

external volume: overlap error with empty catalog?

-werners-
Esteemed Contributor III

Hi all,

I try to create an external volume in a catalog/schema on a subpath of the external location.
This works fine, however for a certain path I cannot create an external volume.  I always get
Input path url overlaps with managed storage within 'CreateVolume' call, for this path only.

However, there is nothing in our catalogs (fresh dev environment). So there cannot be any overlap!
This being a dev environment, we messed around trying unity out, so we are thinking perhaps unity caches some metadata of removed catalogs/tables/...?
Is there any way to find out where the issue could be located?
I already looked in system.information_schema, but there is nothing to be found.

Any thoughts?

1 ACCEPTED SOLUTION

Accepted Solutions

-werners-
Esteemed Contributor III

The issue disappeared by itself after a while (around 30 days) without Databricks support changing anything.
We (myself and databricks) suspect that a garbage collect or something freed locked paths.

View solution in original post

7 REPLIES 7

Kaniz
Community Manager
Community Manager

Hi @-werners-The error message you're seeing suggests that the path you're using for your external volume overlaps with an approach used for managed storage. This can occur if the way you're using for your external book is a subpath of a path used for managed storage. 

Here are a few things you can try to resolve this issue:

1. **Check the path**: Make sure the path you're using for your external volume doesn't overlap with any paths used for managed storage. Even if you're working in a fresh dev environment, it's possible that some managed storage paths were set up previously and are causing this conflict.

2. **Check your permissions**: Ensure you have the necessary permissions to create an external volume. According to the Databricks documentation, you need the CREATE EXTERNAL VOLUME permission for the external location, USE SCHEMA and CREATE VOLUME for the schema, and USE CATALOG for the catalog.

3. **Clear cache**: If you've been experimenting with Unity, it's possible that some metadata is being cached and causing this issue. Try clearing your cache to see if this resolves the issue.

If these suggestions don't help, I recommend reaching out to Databricks support for further assistance by filing a support ticket.

-werners-
Esteemed Contributor III

point 1 and 2 can't be it as I have full permissions on all and the catalog is empty.  Creating a volume on a subpath of this problematic directory works ...
How do I clear this cache?  You mean browser cache or workspace cache?

-werners-
Esteemed Contributor III

I created a support request as this should not occur.

Kaniz
Community Manager
Community Manager
Our support team will look into the matter to determine the root cause and take appropriate action to prevent such occurrences in the future.

-werners-
Esteemed Contributor III

The issue disappeared by itself after a while (around 30 days) without Databricks support changing anything.
We (myself and databricks) suspect that a garbage collect or something freed locked paths.

owlleg6
New Contributor III

I have the same issue. I have created a Catalog using External Location, deleted it and now i can't create Volume for this Location even thought Catalog is deleted long ago

-werners-
Esteemed Contributor III

it is already a few months ago but I created a ticket for this. So the issue resolved itself after a while.
We believe that after 30 days Unity does a cleanup and the deleted items are scrapped completely.
Not sure if that is the case for you though.  If not, I'd create a ticket too (after checking there is not actual overlap of course).

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!