cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Databricks Academy Learners
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Not able to create external table In hive_metastore permanent

harsh_Dev
New Contributor III

Hi @everyone i am using databricks community edition in which i created one database and some external table in the hive_metastore catalog but i terminate the spark cluster then i came back so there was no table could anybody help me with this why its not showing me ? 

i created this way:-

CREATE EXTERNAL TABLE hive_metastore.lending_club.loans_defaulters_delinq
USING PARQUET
LOCATION 'dbfs:/FileStore/tables/cleaned/file_name'

harsh_Dev_0-1744084229445.png

here you can see there is no landing_club database ? why it is happening ?

4 REPLIES 4

WilsonChow
New Contributor II

I think hive_metastore only stores metadata. And Hive registers metadata about the table and its location, but the data files are not managed by Hive.

WilsonChow
New Contributor II

And the community edition, all the created tables would be deleted once the compute being terminated. Because the hive metastore is local to compute. Means like a memory as RAM. Once erased, everything would be gone. As community designed as learn and experimentation usage (limited resource).

Ankit_Hive_Mind
New Contributor II

The tables created via Hive Metastore, stores data in dbfs://user/warehouse/.... but interestingly the Hive metastore by default is local, so it stores metadata related information in the DBFS (which is the attached cluster in Databricks) and since the table is external so data is persisted in cloud storage (S3). Hence deleting the cluster, terminates the Meta data like the Hive table but however since the table is External, so the data is persisted in S3 via DBFS.

shivam_sunny
New Contributor II

To view the tables that have been created, you may need to start the cluster first.

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local communityโ€”sign up today to get started!

Sign Up Now