Unable to make fresh deployments to an agent model serving endpoint due to permission issues

[actualhuman_012]

One of my teammate left and now I cannot make fresh deployments to the agent serving endpoint created by him. Getting below error while trying to deploy with agents.deploy(). Any way to solve it without recreating the endpoint?

PermissionDenied: Principal *************** is not an active member of account ********-****-****-****-************.
Config: host=https://adb-****************.**.azuredatabricks.net, auth_type=runtime

File <command-**************>, line 1
----> 1 deployment_info = agents.deploy(
        model_name=MODEL_NAME_FQN,
        model_version=uc_registered_model_info.version,
        scale_to_zero=False,
        environment_vars=environment_vars,
        workload_size="Small"
    )

File /local_disk0/.ephemeral_nfs/envs/pythonEnv-****/lib/python3.10/site-packages/databricks/sdk/_base_client.py:301, in _BaseClient._perform(...)
    299 error = self._error_parser.get_api_error(response)
    300 if error is not None:
--> 301     raise error from None

[szymon_dybczak]

Hi @actualhuman_012 ,

That a bit problematic. According to docs:

Create custom model serving endpoints | Databricks on AWS
"The identity under which a model serving endpoint runs is tied to the original creator of the endpoint.
After endpoint creation, the associated identity cannot be changed or updated on the endpoint. This identity and its associated permissions are used to access Unity Catalog resources for deployments.
If the identity does not have the appropriate permissions to access the needed Unity Catalog resources, you must delete the endpoint and recreate it under a user or service principal that can access those Unity Catalog resources."

So you cannot transfer ownership - recreating the endpoint is the official path. 

To avoid this situation again, always create serving endpoints under a service principal rather than a personal user account. That way, when someone leaves the team, the endpoint's identity remains valid. This is the standard best practice for any production workload in Databricks 🙂

If the answer was helpful, please consider marking it as accepted solution

[actualhuman_012]

Thanks @szymon_dybczak , I wonder what are the things to take care of during re-creation. Is there any official documentation? Don't want to break analytics pipelines or Lakehouse monitoring LLM as Judge pipelines. Don't know how it will affect the payload tables.