cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results for 
Search instead for 
Did you mean: 

READ FILES and WRITE FILES when using Hive Metastore

knawara
Contributor

Hello,

I'm confused about documentation on privilege types when using HMS.

The following page is supposed to talk about HMS

https://docs.databricks.com/sql/language-manual/sql-ref-privileges-hms.html

but it also mentions 

READ FILES

Query files directly using the storage credential or external location.

WRITE FILES

Directly COPY INTO files governed by the storage credential or external location.

If I understand correctly these (Storage Credential and External Location) only apply to Unity Catalog, as per this page:

https://docs.databricks.com/sql/language-manual/sql-ref-external-locations.html

Is this a mistake in a documentation or there is something more fundamental that I don't understand?

7 REPLIES 7

Lakshay
Esteemed Contributor
Esteemed Contributor

Hi @Chris Nawara​ , The Privilege types and Secure objects are available both in HMS and Unity Catalog. However, there is a difference in implementation across both of them. And as the document mentions "The privilege model and securable objects differ depending on whether you are using a Unity Catalog metastore or the legacy Hive metastore"

HI @Lakshay Goel​ , thanks for the rapid response!

There are two pages in the documentation, one for HMS:

https://docs.databricks.com/sql/language-manual/sql-ref-privileges-hms.html

which claims "This article describes the privilege model for the legacy Hive metastore".,

and one for Unity Catalog:

https://docs.databricks.com/sql/language-manual/sql-ref-privileges.html

This article describes the privilege model for the Unity Catalog.

READ/WRITE FILES are mentioned in both. What I want to clarify is:

  1. Is READ/WRITE FILES a Unity-Catalog-only concept?
  2. If not, what is its meaning when working with HMS?

Lakshay
Esteemed Contributor
Esteemed Contributor

Hi @Chris Nawara​ , The two documentations talk about data governance. The concept of data governance is not exclusive to Unity Catalog. The difference here is that Unity Catalog helps you in implementing Data Governance at a much more granular level and better than HMS. So, to answer your questions

  1. Is READ/WRITE FILES a Unity-Catalog-only concept? No
  2. If not, what is its meaning when working with HMS? You can read/write a file with both HMS and UC. But how the data governance and security works in two is the difference.

Hi @Lakshay Goel​ ,

I'm not talking about reading/writing files, but about READ FILES/WRITE FILES permission that can be granted e.g. in the following way:

GRANT READ FILES ON STORAGE CREDENTIAL <storage_credential_name> TO <principal>;

(from https://docs.databricks.com/data-governance/unity-catalog/manage-external-locations-and-credentials....

As you said, that's a governance question and some things are done way better in UC than in HMS (but for certain reasons not dependent on me UC is not an option). But there are differences between the two, so I guess my question is whether I can use this construct with both HMS and UC, or with UC only

Anonymous
Not applicable

Hi @Chris Nawara​ 

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. 

We'd love to hear from you.

Thanks!

Hi @Vidula Khanna​ , thanks for checking in! Not yet, my last message is still unanswered

Anonymous
Not applicable

Hi @Chris Nawara​ 

 I'm sorry you could not find a solution to your problem in the answers provided.

Our community strives to provide helpful and accurate information, but sometimes an immediate solution may only be available for some issues.

I suggest providing more information about your problem, such as specific error messages, error logs or details about the steps you have taken. This can help our community members better understand the issue and provide more targeted solutions.

Alternatively, you can consider contacting the support team for your product or service. They may be able to provide additional assistance or escalate the issue to the appropriate section for further investigation.

Thank you for your patience and understanding, and please let us know if there is anything else we can do to assist you.

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!