cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

The database and tables disappears when I delete the cluster

William_Scardua
Valued Contributor

Hi guys,

I have a trial databricks account, I realized that when I shutdown the cluster my databases and tables is disappear .. that is correct or thats is because my account is trial ?

5 REPLIES 5

Prabakar
Databricks Employee
Databricks Employee

Hi @William Scarduaโ€‹ this is expected. To view the database and tables you need a running cluster. If none of the clusters are running then you won't be able to see the database list and you will see as below.

image 

Once you start a cluster you will be able to see the database and tables.

image 

Hubert-Dudek
Esteemed Contributor III

exactly as @Prabakar Ammeappinโ€‹ said. If you want to keep access to your databases you need general-purpose cluster running. It will not work with (cheaper) job cluster. When you use jdbc/odbc connector to Spark (or some built in like in PowerBI) you also need to have general-purpose cluster running but soon for that purpose new SQL cluster will be introduced (which will be much cheaper than general-purpose cluster).

William_Scardua
Valued Contributor

Thank you @Hubert Dudekโ€‹ and @Prabakar Ammeappinโ€‹ 

โ€‹

but it's a bit weird because if i shut down my cluster I lose all my databases and tables

โ€‹

I create a new cluster and a new database

Screen Shot 2021-12-02 at 08.13.04I shutting down/delete my cluster

Screen Shot 2021-12-02 at 08.13.36When I create a new cluster the database is not existe anymore

Screen Shot 2021-12-02 at 08.17.17โ€‹

Can you imagine, if somebody shutting down the cluster by accident or beat the idle time ... in this case you organisation lose all of databases and tables. If that's true, does not worth using the databases in databricks but in a other cloud

โ€‹

โ€‹

โ€‹

Hubert-Dudek
Esteemed Contributor III

If it is community edition free edition it can happen as it is only for education purposed,

In normal edition all data is stored on your blob storage so whatever happened it can not be lost, additionally default storage is delta so it is even transactional. Temporary views are also possible to create.

You can send more details about your edition and code so I will be glad to help.

Additionally you can create database or table using specified location. The best practice is to use own storage and mount it to directory so you will have full control.

Prabakar
Databricks Employee
Databricks Employee

@William Scarduaโ€‹ if it's an external hive metastore or Glue catalog you might be missing the configuration on the cluster.

https://docs.databricks.com/data/metastores/index.html

Also as mentioned by @Hubert Dudekโ€‹ , if it's a community edition then this is possible.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group