Databricks SQL connectivity in Python with Service Principals

harripy
Databricks Partner

Tried to use M2M OAuth connectivity on Databricks SQL Warehouse in Python:

from databricks.sdk.core import Config, oauth_service_principal
from databricks import sql

....
config = Config(host=f"https://{host}",
                                client_id=db_user,
                                client_secret=passwd)
oauth_sp = oauth_service_principal(config)
return sql.connect(server_hostname=host,
                   http_path=http_path,
                   catalog=catalog,
                   schema=schema,
                   credentials_provider=oauth_sp)

  The params mentioned are properly assigned, but this fails to an error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/path/venv/lib/python3.11/site-packages/databricks/sql/__init__.py", line 50, in connect
    return Connection(server_hostname, http_path, access_token, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/path/venv/lib/python3.11/site-packages/databricks/sql/client.py", line 189, in __init__
    self._session_handle = self.thrift_backend.open_session(
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/path/venv/lib/python3.11/site-packages/databricks/sql/thrift_backend.py", line 464, in open_session
    response = self.make_request(self._client.OpenSession, open_session_req)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/path/venv/lib/python3.11/site-packages/databricks/sql/thrift_backend.py", line 393, in make_request
    self._handle_request_error(error_info, attempt, elapsed)
  File "/path/venv/lib/python3.11/site-packages/databricks/sql/thrift_backend.py", line 261, in _handle_request_error
    raise network_request_error
databricks.sql.exc.RequestError: Error during request to server

The credentials provider seems to be set correctly, yet the connection is tried to be established with the access_token - what am I missing?