04-26-2024 12:35 AM
Hi all,
I am getting an error when I try to 'Browse' an external location in catalog explorer through Databricks UI.
This is the error I get:
04-26-2024 12:42 AM
Hi Sean,
This is expected for a managed path, as a user, we are not supposed to directly access paths that are managed by UC.
Hope you are able to access this KB with detailed info - https://kb.databricks.com/unity-catalog/invalid_parameter_valuelocation_overlap-overlaps-with-manage...
Thanks!
04-29-2024 11:00 AM
04-26-2024 12:42 AM
Hi Sean,
This is expected for a managed path, as a user, we are not supposed to directly access paths that are managed by UC.
Hope you are able to access this KB with detailed info - https://kb.databricks.com/unity-catalog/invalid_parameter_valuelocation_overlap-overlaps-with-manage...
Thanks!
08-19-2024 07:25 AM
Hi NandiniN,
I'm having the same error. Let me give you some context of my use case.
I need to calculate the size of the container, so I created a function that loops over the folder structure using "dbutils.fs.ls". Inside container we have both, external and managed tables. But I'm not attempting to create anything inside container, just need to read every file size inside container. When attempting to do so, is that I'm getting below error.
===== ERROR =====
[RequestId=887e131d-4cea-41f6-a226-7b804e07f938 ErrorClass=INVALID_PARAMETER_VALUE.LOCATION_OVERLAP] Input path url 'abfss://<container>@<storage_account>.dfs.core.windows.net/managed/<subfolder>/data/__unitystorage' overlaps with managed storage within 'ListFiles' call.
I've assigned permissions "Storage Blob Data Contributor" + "Storage Blob Data Owner" on storage account to the Managed Identity declared in Catalog. I've also added permissions "ALL PRIVILEGES" on External location and Storage Credentials. But I'm still getting the same error.
Is this supported in UC? If this is not the best approach, how would you achieve this task?
Regards, Emiliano
04-29-2024 11:00 AM
04-29-2024 12:08 PM
Adding some tips...
It seems that you have created a metastore with root path definition (metastore-level managed storage). If you're planning to deploy a multi-workspace environment (Development, QA and Production), I strongly recommend you to change this configuration to Catalog-level storage.
Since in general those kind of environments should be isolated, you will have tough days as soon as it gets bigger when using metastore-level storage.
04-29-2024 02:08 PM
Hi @gmiguel ,
Thank you for the advice.
Is this a setting when creating the metastore, or do I simply not specify an external location for metastore and only do so for catalogs?
Thanks,
Sean
04-29-2024 03:14 PM
Simply do not specify a location during metastore creation. Do it at catalog level and schema level (optional).
a week ago - last edited a week ago
I'm running into this error when only using Autoloader within my UC. So, my assumption is that when Autoloader does a file listing for my source files, it creates the error. I've tried pointing Autloloader to an s3 location (a public bucket) and also a /Volume path that is mapped into my unity catalog. Both generate the same error.
I have no problems with a similar attempt outside of UC using my hive_metastore which I share with my AWS instance.
My metastore path is unspecified but my catalog path is specified.
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group