Migrating from Databricks Notebooks to IDE for Development

[Raymond_Garcia]

Hello, we are developers who have been creating a system in Databricks with Scala. We enabled the Git feature, so the project is in a repository. 

The project has a lot of notebooks and a lot of calls to other notebooks. Sometimes it is a little overwhelmed. Debugging is challenging. I miss the IDE's features. This project is in production, and we want to migrate it to an IDE. I have some questions.

What should be the best starting point for migrating this project?.

Should I create a project from scratch or use the same repository?. 

Are there best practices to move from notebooks to IDE? If so, could you please provide the link with some examples?

Are there any examples that we could use as references?

I really appreciate any help you can provide.

Thanks!

[Hubert-Dudek]

@Raymond Garcia​ , It is almost impossible to develop in IDE out of databricks as you can not debug anything.

Soon Databricks Tunnel for VS Code will be available, so it should be possible.

[Raymond_Garcia]

Thanks!

[Raymond_Garcia]

Hello @Kaniz Fatma​ We are in the process of migrating from Databricks Notebooks to IDE. if you don't mind, I will leave to ticket open to respond to my question once we finish the migration 

[Raymond_Garcia]

it is true that we can't work without data bricks but we can develop an IDE and send the jar to databricks, this will allow us to create unit tests, and use the IDE capabilities (i.e fast navigation among classes).