How can I backup my Databricks instance?

[ChristianWuerdi]

We have a Databricks instance on Azure that has somewhat organically grow with dozens of users and hundreds of notebooks. How do I conveniently backup this env so in case disaster strikes the notebooks aren't lost? The data itself is backed by Azure storage accounts so that's already taken care of.

[-werners-]

I use the Databricks CLI for that and export the entire workspace on a regular basis.

Check it out, it has tons of handy functions.

https://docs.microsoft.com/en-us/azure/databricks/dev-tools/cli/

[ChristianWuerdi]

Ok thanks, bit uncomfortable to rely on a tool that's labelled experimental to do backups but I guess it's the best option at the moment. You wouldn't happen to have some handy command line example to backup a whole workspace?

[-werners- · Accepted solution]

Totally understandable.

But why don't you use the new Repos functionality? Like that you always have your code in git.

(the cli command is:

databricks workspace export_dir SOURCE_PATH TARGET PATH

Sourcepath is "/" for the whole workspace.)

But Repos is a way better alternative, no idea why it did not pop into my head yesterday.

[Kaniz]

Hi @Christian Wuerdig​ , Just a friendly follow-up. Do you still need help, or @Werner Stinckens​ (Customer)​ 's response help you to find the solution? Please let us know.

[ChristianWuerdi]

@Kaniz Fatma​ All good thanks, combination of CLI + gradually migrating everything to git is a viable solution