I have created a model serving endpoint in Databricks. I'd like to now update this endpoint programmatically from a workflow task.
Following instructions on this page:
https://docs.databricks.com/machine-learning/model-serving/create-manage-serving-endpoints.html
I have written a notebook task to update the endpoint. The notebook runs fine on its own (when I run it cell by cell manually), but it fails when run as part of a workflow.
Specifically, the GET request to get the status of the model endpoint fails with the following ConnectionError exception:
ConnectionError: HTTPSConnectionPool(host='none', port=443): Max retries exceeded with url: /api/2.0/serving-endpoints/xxx (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -2] Name or service not known'))
The same exact GET request works when the notebook runs on its own. I have played with different tokens with no success.
It appears that the APIs to manage model serving endpoints aren't accessible from workflow tasks.
Can anyone help resolve this?
