โ12-01-2021 04:13 PM
Hi guys,
I have a trial databricks account, I realized that when I shutdown the cluster my databases and tables is disappear .. that is correct or thats is because my account is trial ?
โ12-02-2021 12:17 AM
Hi @William Scarduaโ this is expected. To view the database and tables you need a running cluster. If none of the clusters are running then you won't be able to see the database list and you will see as below.
Once you start a cluster you will be able to see the database and tables.
โ12-02-2021 02:38 AM
exactly as @Prabakar Ammeappinโ said. If you want to keep access to your databases you need general-purpose cluster running. It will not work with (cheaper) job cluster. When you use jdbc/odbc connector to Spark (or some built in like in PowerBI) you also need to have general-purpose cluster running but soon for that purpose new SQL cluster will be introduced (which will be much cheaper than general-purpose cluster).
โ12-02-2021 03:59 AM
Thank you @Hubert Dudekโ and @Prabakar Ammeappinโ
โ
but it's a bit weird because if i shut down my cluster I lose all my databases and tables
โ
I create a new cluster and a new database
I shutting down/delete my cluster
When I create a new cluster the database is not existe anymore
โ
Can you imagine, if somebody shutting down the cluster by accident or beat the idle time ... in this case you organisation lose all of databases and tables. If that's true, does not worth using the databases in databricks but in a other cloud
โ
โ
โ
โ12-02-2021 04:20 AM
If it is community edition free edition it can happen as it is only for education purposed,
In normal edition all data is stored on your blob storage so whatever happened it can not be lost, additionally default storage is delta so it is even transactional. Temporary views are also possible to create.
You can send more details about your edition and code so I will be glad to help.
Additionally you can create database or table using specified location. The best practice is to use own storage and mount it to directory so you will have full control.
โ12-02-2021 05:04 AM
@William Scarduaโ if it's an external hive metastore or Glue catalog you might be missing the configuration on the cluster.
https://docs.databricks.com/data/metastores/index.html
Also as mentioned by @Hubert Dudekโ , if it's a community edition then this is possible.
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโt want to miss the chance to attend and share knowledge.
If there isnโt a group near you, start one and help create a community that brings people together.
Request a New Group