READ FILES and WRITE FILES when using Hive Metastore
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-15-2023 04:33 AM
Hello,
I'm confused about documentation on privilege types when using HMS.
The following page is supposed to talk about HMS
https://docs.databricks.com/sql/language-manual/sql-ref-privileges-hms.html
but it also mentions
READ FILES
Query files directly using the storage credential or external location.
WRITE FILES
Directly COPY INTO files governed by the storage credential or external location.
If I understand correctly these (Storage Credential and External Location) only apply to Unity Catalog, as per this page:
https://docs.databricks.com/sql/language-manual/sql-ref-external-locations.html
Is this a mistake in a documentation or there is something more fundamental that I don't understand?
- Labels:
-
Hive metastore
-
Unity Catalog
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-15-2023 04:56 AM
Hi @Chris Nawara , The Privilege types and Secure objects are available both in HMS and Unity Catalog. However, there is a difference in implementation across both of them. And as the document mentions "The privilege model and securable objects differ depending on whether you are using a Unity Catalog metastore or the legacy Hive metastore"
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-15-2023 05:11 AM
HI @Lakshay Goel , thanks for the rapid response!
There are two pages in the documentation, one for HMS:
https://docs.databricks.com/sql/language-manual/sql-ref-privileges-hms.html
which claims "This article describes the privilege model for the legacy Hive metastore".,
and one for Unity Catalog:
https://docs.databricks.com/sql/language-manual/sql-ref-privileges.html
This article describes the privilege model for the Unity Catalog.
READ/WRITE FILES are mentioned in both. What I want to clarify is:
- Is READ/WRITE FILES a Unity-Catalog-only concept?
- If not, what is its meaning when working with HMS?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-17-2023 03:08 AM
Hi @Chris Nawara , The two documentations talk about data governance. The concept of data governance is not exclusive to Unity Catalog. The difference here is that Unity Catalog helps you in implementing Data Governance at a much more granular level and better than HMS. So, to answer your questions
- Is READ/WRITE FILES a Unity-Catalog-only concept? No
- If not, what is its meaning when working with HMS? You can read/write a file with both HMS and UC. But how the data governance and security works in two is the difference.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-17-2023 08:12 AM
Hi @Lakshay Goel ,
I'm not talking about reading/writing files, but about READ FILES/WRITE FILES permission that can be granted e.g. in the following way:
GRANT READ FILES ON STORAGE CREDENTIAL <storage_credential_name> TO <principal>;
As you said, that's a governance question and some things are done way better in UC than in HMS (but for certain reasons not dependent on me UC is not an option). But there are differences between the two, so I guess my question is whether I can use this construct with both HMS and UC, or with UC only
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-21-2023 02:16 AM
Hi @Chris Nawara
Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help.
We'd love to hear from you.
Thanks!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-21-2023 03:10 AM
Hi @Vidula Khanna , thanks for checking in! Not yet, my last message is still unanswered
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-10-2023 06:01 PM
Hi @Chris Nawara
I'm sorry you could not find a solution to your problem in the answers provided.
Our community strives to provide helpful and accurate information, but sometimes an immediate solution may only be available for some issues.
I suggest providing more information about your problem, such as specific error messages, error logs or details about the steps you have taken. This can help our community members better understand the issue and provide more targeted solutions.
Alternatively, you can consider contacting the support team for your product or service. They may be able to provide additional assistance or escalate the issue to the appropriate section for further investigation.
Thank you for your patience and understanding, and please let us know if there is anything else we can do to assist you.