Good morning,

For information the error is not at all related to the limitations of databricks connect.

After various tests, in my case, it turns out that it is necessary to update the libraries of the venv used with databricks connect.

Here are the python library updates I made:

  • databricks-connect from 10.4.12 to 10.4.21
  • databricks-cli from 0.17.3 to 0.17.4
  • mlflow from 1.26.1 to 2.2.1
  • protobuf from 3.20.0 to 3.20.3

Note that I work with a 10.4 lts cluster

After these updates, the code example above works fine on intelliJ with databricks connect