I agree with most of the comments above, that the current approach of databricks-connect is not great (it sucks to be frankly). Its an issue that was bugging me since more than 2 years now.
By the way, i checked how this could be done with poetry and uv. In poetry i got something somewhat working since this PR was merged: https://github.com/python-poetry/poetry/pull/9553
With uv I still don't have an acceptable solution. In fact the uv devs seem to claim, that there is no way to solve this with the current python packaging specifications: https://github.com/astral-sh/uv/issues/10238#issuecomment-2575893989