Hi, 

You can use any of existing IDE. I'm using pycharm. I have created my own utils to run code on databricks. In .env file I have environmental variables and using SDK I'm creating SparkSession object and WorkspaceObject that you can use to read credentials, etc. (gives you dbutils functionality). In your pyproject.toml you need only databricks connect library (version for your cluster runtime).

"databricks-connect == 15.4.6",

#ENV Variables for Databricks SDK
DATABRICKS_HOST =https://....
DATABRICKS_TOKEN=xxxxx
CLUSTER_ID=xxxx