cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results for 
Search instead for 
Did you mean: 

Models signature inaccessible in Unity Catalog

owlleg6
New Contributor III

I am using Unity Catalog to store my models. My Unity Catalog meta-store is hosted in an Azure Storage Account, which has public network access disabled. Access is restricted to certain IP ranges and private endpoint connections only.

Recently, my development team has noticed that the signatures of the models are inaccessible, although we can access every table, volume, etc., using the Private Endpoint connected to the Storage Account. Interestingly, if I open the Storage Account to the public network, the model signatures become accessible. Could you please explain why all folders within the Unity Catalog meta-store are accessible except for the 'models' folder?

6 REPLIES 6

owlleg6
New Contributor III

MicrosoftTeams-image (9).png

This exception from mlflow when downloading artifact

Kaniz
Community Manager
Community Manager

Hi @owlleg6 , Thank you for sharing your scenario regarding Unity Catalog, Azure Storage Account, and private endpoints. 

 

Let’s explore the intricacies of this situation:

 

Unity Catalog and Azure Storage:

  • Unity Catalog serves as an essential component in Databricks data governance. It manages data assets such as tables, views, and volumes, along with permissions governing access.
  • Your Unity Catalog meta-store resides in an Azure Storage Account. This storage account has public network access disabled, ensuring security by restricting access to specific IP ranges and private endpoints.

Private Endpoints and Model Signatures:

  • Private endpoints provide additional security and control over network traffic to and from your storage account.
  • When you create a private endpoint for your storage account, all traffic to and from the storage account is routed through this endpoint. Clients and applications must use the private endpoint to connect.
  • Now, here’s the interesting part: While other folders within the Unity Catalog meta-store are accessible via the private endpoint, the ‘models’ folder remains elusive.

Potential Reasons for Inaccessibility:

  • The specific reason for the inaccessibility of the ‘models’ folder might involve how the Unity Catalog interacts with the storage account.
  • It’s possible that the models’ folder contains sensitive information or requires a different level of access control.
  • Consider checking the permissions, ACLs, and security settings specifically for the ‘models’ folder. There might be unique configurations affecting its accessibility.

Performance and Latency Considerations:

  • While private endpoints enhance security, they can introduce additional latency or network overhead. The impact depends on your network configuration and client locations.
  • Evaluate the trade-offs between security and performance. The private endpoint ensures data exfiltration prevention but may affect response times.

Testing and Configuration:

If you need further assistance, feel free to ask!

owlleg6
New Contributor III

Hello, your moderators are deleting my previous replies, that's funny something is not working properly, i am trying to find out the solution but you are deleting the posts.
So, the issue is with a networking for sure, not with the permissions. Where can i raise a ticket? This is critical issue for my team, we have to disable public access to Storage Account.

Cheers

Rikard007
New Contributor II

We are facing the same issue. We can’t reconsider security setup since that is enforced by IT security. Does anyone have suggestions on how to resolve this issue so we can save and access the signatures from models stored in UC? 

LuisBronchal
New Contributor II

I have the same problem. It must be a permissions issue. In my case, it happens when the models are under a schema that was created without indicating a specific "MANAGED LOCATION". In those cases, I also have problems trying to load a model. When the models are under a schema created with a particular "MANAGED LOCATION" all works ok for me.

owlleg6
New Contributor III

Fair point, what if i don't want to use External Locations for Schemas where models will be stored? Why default Metastore location does not allow accessing signatures.

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!