a month ago
Hi everyone,
I was trying to understand: when a managed table is created, Databricks stores the metadata in the Hive metastore and the data in the cloud storage managed by it, which in the case of Azure Databricks will be an Azure Storage Account. But i don’t see this storage account from the Azure portal?
The second question is: besides the fact that if the table is managed, and if the table or the Azure Databricks resource is deleted, both the data and the metadata are lost, does the same apply to clusters? So, if I create a managed table TEST with cluster A and i delete the cluster A, and then recreate cluster B, do I still see the table TEST or is it lost in this case as well?
a month ago
@YOUKE
Are you trying to check the storage location of the schema?
Managed tables in Hive metastore are always created in the storage location of the containing schema.
https://learn.microsoft.com/en-us/azure/databricks/database-objects/hive-metastore
Regarding your other question. The tables are not cluster specific. If you have created a managed table from cluster A it will be accessible from cluster B as well if you have enough privilege to access the table.
a month ago
@YOUKE
Are you trying to check the storage location of the schema?
Managed tables in Hive metastore are always created in the storage location of the containing schema.
https://learn.microsoft.com/en-us/azure/databricks/database-objects/hive-metastore
Regarding your other question. The tables are not cluster specific. If you have created a managed table from cluster A it will be accessible from cluster B as well if you have enough privilege to access the table.
a month ago
Yes, exactly. I’m trying to understand where they are stored, but it seems to me that they are present in the Hive Warehouse, which should be the storage managed by Azure Databricks using Apache Hive, it's correct?
a month ago - last edited a month ago
Hi @YOUKE ,
As @MuthuLakshmi pointed out, managed tables in Hive metastore are always created in the storage location of the containing schema. So, if you don’t provide a location when creating schema in hive_metastore, the default location user/hive/warehouse/schema-name is used. This location is on the DBFS root, which is not recommended for storing any production data.
3 weeks ago
Thank you so much for the solution.
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group