topic Re: Will the database be deleted when the cluster terminates ? in Data Engineering

Will the database be deleted when the cluster terminates ?

Logan_Data_Inc_ — Mon, 20 Dec 2021 20:07:44 GMT

Hello,

I'm working in Databricks Community Edition. So I will terminate my cluster after my work because anyways it will be terminated after 2 hours. I'm creating a database to store all my transformed data. Will the database be deleted when I terminate the cluster ?

I have tried it and seems like it will delete the database because it doesn't recognize the database name I gave.

Error Message - AnalysisException: Database 'db_name' not found

But I can see my database path in DBFS: dbfs:/user/hive/warehouse/db_name.db.

When I create a new database and try to add data to tables, then there is an error saying that the table already exists and not empty.

Please help me with this problem.

Re: Will the database be deleted when the cluster terminates ?

Hubert-Dudek — Mon, 20 Dec 2021 20:21:09 GMT

Problem is that it is community edition so metastore with databases didn't survive.

please try:

CREATE DATABASE IF NOT EXISTS db_name LOCATION 'dbfs:/user/hive/warehouse/db_name.db'

Re: Will the database be deleted when the cluster terminates ?

Logan_Data_Inc_ — Mon, 20 Dec 2021 20:37:46 GMT

Thanks for the answer Hubert 😊

Re: Will the database be deleted when the cluster terminates ?

Anonymous — Mon, 20 Dec 2021 21:15:00 GMT

@Hubert Dudek - Thanks for answering so quickly!!

@Sriram Devineedi - If Hubert's answer solved the issue for you, would you be happy to mark his answer as best? That helps others know where to look.