- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-06-2022 02:39 PM
Hi @karthik p ,
I have to disagree.
Managed tables are the default way to create tables in Unity Catalog. These tables are stored in the Unity Catalog root storage location that you configured when you created a metastore. Databricks recommends using managed tables whenever possible to ensure support of Unity Catalog features. All managed tables use Delta Lake.
source: https://docs.databricks.com/data-governance/unity-catalog/best-practices.html#organize-your-data
@Ashley Betts if you see this:
Each metastore is configured with a root storage location, which is used for managed tables. You need to ensure that no users have direct access to this storage location. Giving access to the storage location could allow a user to bypass access controls in a Unity Catalog metastore and disrupt auditability. For these reasons, you should not reuse a bucket that is your current DBFS root file system or has previously been a DBFS root file system for the root storage location in your Unity Catalog metastore.
I don't have resources on the UC backup. If you read the above you can find out that Unity Catalog / metastore managed tables are stored in the metastore root bucket.
You should Create an IAM role that Databricks uses to give access to that storage bucket, so basically there shouldn't be other mechanisms to read/write data (outside the databricks) to make sure the data won't get corrupted, or someone will bypass the access control set in Unity Catalog. When you use Delta Tables you can use Time travel to restore the previous version of the tables.
Backup seems tricky as managed tables are no longer stored in locations corresponding to the names, but they have some sort of uuid and I think the mapping of the table name to the location is stored in the Databricks control plane (database/backend).
I have always liked external tables, but with the UC I am leaning more towards managed tables.
thanks,
Pat.