Databricks

William_Scardua · ‎12-01-2021

Hi guys,

I have a trial databricks account, I realized that when I shutdown the cluster my databases and tables is disappear .. that is correct or thats is because my account is trial ?

Prabakar · ‎12-02-2021

Hi @William Scardua this is expected. To view the database and tables you need a running cluster. If none of the clusters are running then you won't be able to see the database list and you will see as below.

Once you start a cluster you will be able to see the database and tables.

Hubert-Dudek · ‎12-02-2021

exactly as @Prabakar Ammeappin said. If you want to keep access to your databases you need general-purpose cluster running. It will not work with (cheaper) job cluster. When you use jdbc/odbc connector to Spark (or some built in like in PowerBI) you also need to have general-purpose cluster running but soon for that purpose new SQL cluster will be introduced (which will be much cheaper than general-purpose cluster).

William_Scardua · ‎12-02-2021

Thank you @Hubert Dudek and @Prabakar Ammeappin

but it's a bit weird because if i shut down my cluster I lose all my databases and tables

I create a new cluster and a new database

I shutting down/delete my cluster

When I create a new cluster the database is not existe anymore

Can you imagine, if somebody shutting down the cluster by accident or beat the idle time ... in this case you organisation lose all of databases and tables. If that's true, does not worth using the databases in databricks but in a other cloud

Hubert-Dudek · ‎12-02-2021

If it is community edition free edition it can happen as it is only for education purposed,

In normal edition all data is stored on your blob storage so whatever happened it can not be lost, additionally default storage is delta so it is even transactional. Temporary views are also possible to create.

You can send more details about your edition and code so I will be glad to help.

Additionally you can create database or table using specified location. The best practice is to use own storage and mount it to directory so you will have full control.