External locations

[Snoonan]

Hi all,

I am getting an error when I try to 'Browse' an external location in catalog explorer through Databricks UI.

This is the error I get:

Error loading files.

Input path url 'abfss://dbx-ccc-hive-metastore-unity-catalog@xxxxxxxxxx.dfs.core.windows.net/__unitystorage' overlaps with managed storage within 'ListFiles' call

Has anyone seen this error before and/or understands what is going wrong?

Thanks,

Sean

[NandiniN · Accepted solution]

Hi Sean,

This is expected for a managed path, as a user, we are not supposed to directly access paths that are managed by UC.

Hope you are able to access this KB with detailed info - https://kb.databricks.com/unity-catalog/invalid_parameter_valuelocation_overlap-overlaps-with-manage...

Thanks!

[Snoonan · Accepted solution]

Hi @NandiniN ,

Thank you for quashing my concerns.

Thanks,

Sean

[gmiguel]

Adding some tips...
It seems that you have created a metastore with root path definition (metastore-level managed storage). If you're planning to deploy a multi-workspace environment (Development, QA and Production), I strongly recommend you to change this configuration to Catalog-level storage. 

Since in general those kind of environments should be isolated, you will have tough days as soon as it gets bigger when using metastore-level storage.

[Snoonan]

Hi @gmiguel ,

Thank you for the advice.

Is this a setting when creating the metastore, or do I simply not specify an external location for metastore and only do so for catalogs?

Thanks,

Sean

[gmiguel]

Simply do not specify a location during metastore creation. Do it at catalog level and schema level (optional).