- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-25-2021 05:37 AM
yes.
one way is to develop everything locally on your pc, so you also need to have spark installed.
This is of course not ideal as you will not have some interesting stuff that databricks provides.
But it can be done. What you have to do is create a whl and put that on databricks.
A much better way imo is to use databricks-connect. This tool enables you to use your IDE (pycharm) and let the code run on databricks, not locally:
https://docs.microsoft.com/en-us/azure/databricks/dev-tools/databricks-connect
Mind that the databricks-connect devs don't follow the ultrafast release schedule of the databricks platform.
Since a short while you can also use Databricks repos for modules and functions:
https://docs.microsoft.com/en-us/azure/databricks/repos#work-with-python-and-r-modules