How to access model endpoint management APIs from workflow tasks?

fijoy — Tue, 11 Jul 2023 16:21:31 GMT

I have created a model serving endpoint in Databricks. I'd like to now update this endpoint programmatically from a workflow task.

Following instructions on this page:

https://docs.databricks.com/machine-learning/model-serving/create-manage-serving-endpoints.html

I have written a notebook task to update the endpoint. The notebook runs fine on its own (when I run it cell by cell manually), but it fails when run as part of a workflow.

Specifically, the GET request to get the status of the model endpoint fails with the following ConnectionError exception:

ConnectionError: HTTPSConnectionPool(host='none', port=443): Max retries exceeded with url: /api/2.0/serving-endpoints/xxx (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -2] Name or service not known'))

The same exact GET request works when the notebook runs on its own. I have played with different tokens with no success.

It appears that the APIs to manage model serving endpoints aren't accessible from workflow tasks.

Can anyone help resolve this?

topic How to access model endpoint management APIs from workflow tasks? in Get Started Discussions

How to access model endpoint management APIs from workflow tasks?