This issue seems to be a classic rulebook of muddying waters and killing open source innovation (but using the opensource behind for own advantage). Whoever reads this should take into account the unfair advantage (hidden layers and layers underneath) that databricks-connect package is trying to do by being incompatible with pyspark.... Not good !