- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-21-2021 03:30 PM
We have a Databricks instance on Azure that has somewhat organically grow with dozens of users and hundreds of notebooks. How do I conveniently backup this env so in case disaster strikes the notebooks aren't lost? The data itself is backed by Azure storage accounts so that's already taken care of.
- Labels:
-
Administration
-
Azure
-
Backup
-
Databricks Instance
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-22-2021 01:35 AM
I use the Databricks CLI for that and export the entire workspace on a regular basis.
Check it out, it has tons of handy functions.
https://docs.microsoft.com/en-us/azure/databricks/dev-tools/cli/
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-22-2021 04:59 PM
Ok thanks, bit uncomfortable to rely on a tool that's labelled experimental to do backups but I guess it's the best option at the moment. You wouldn't happen to have some handy command line example to backup a whole workspace?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-23-2021 12:13 AM
Totally understandable.
But why don't you use the new Repos functionality? Like that you always have your code in git.
(the cli command is:
databricks workspace export_dir SOURCE_PATH TARGET PATH
Sourcepath is "/" for the whole workspace.)
But Repos is a way better alternative, no idea why it did not pop into my head yesterday.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-18-2022 03:28 PM
@Kaniz Fatma All good thanks, combination of CLI + gradually migrating everything to git is a viable solution
