I am able to use vscode extension + databricks connect to develop Notebooks on my local computer and run them on my databricks cluster. However I can not figure out how to develop the Notebooks that have the file `.py` extension but identified by Databricks as notebooks by their header
# Databricks notebook source
# COMMAND ----------
import pytest
import os
import sys
My workflow looks like this at the moment:
- develop the notebook on Databrick UI
- push the code to Gitlab
- (optional) pull the code from gitlab on my local computer, develop, push back, pull from databricks repo
The problem with local development is Databricks Connect can only detect Notebooks as such when they have .ipynb extension. I wat to be able to develop and run my code on my local computer the same way I do it Databricks UI.