-werners-
Esteemed Contributor III

yes.

one way is to develop everything locally on your pc, so you also need to have spark installed.

This is of course not ideal as you will not have some interesting stuff that databricks provides.

But it can be done. What you have to do is create a whl and put that on databricks.

A much better way imo is to use databricks-connect. This tool enables you to use your IDE (pycharm) and let the code run on databricks, not locally:

https://docs.microsoft.com/en-us/azure/databricks/dev-tools/databricks-connect

Mind that the databricks-connect devs don't follow the ultrafast release schedule of the databricks platform.

Since a short while you can also use Databricks repos for modules and functions:

https://docs.microsoft.com/en-us/azure/databricks/repos#work-with-python-and-r-modules

View solution in original post