Not able to configure cluster settings instance type using mlflow api 2.0 to enable model serving.

[shane]

I'm able to enable model serving by using the mlflow api 2.0 with the following code...

instance = f'https://{workspace}.cloud.databricks.com'
headers = {'Authorization': f'Bearer {api_workflow_access_token}'} 
 
# Enable Model Serving 
import requests
url = f'{instance}/api/2.0/mlflow/endpoints/enable'
requests.post(url, headers=headers, json={"registered_model_name": f'{model_name}'})

However this automatically sets the cluster setting instance type to be m5a.xlarge, which I DO NOT want it to be. I can manually go into the settings on the UI and change it to be m4.large but I want to be able to do this within the api code above so that I don't have to manually go into the settings and change it.

[Debayan]

Hi, Is this a recent issue or happened earlier? Could you please try directly with the API endpoint from another tool? Also, all deployed versions of a registered model share the same cluster, please check if there is a serving cluster existing already.

Please refer: https://docs.databricks.com/mlflow/model-serving.html#manage-served-versions

[shane]

This is a recent issue, happening right now. There is no serving cluster already existing for this model as it is the only deployed version. I am open to using whatever tool needed to adjust the API endpoint but there is no documentation on what code to pass the endpoint to tell it which type of cluster to start. I can only change the cluster instance type from the UI which is not what I want to do. I want to be able to spin up and down serving clusters from an automated notebook and never have to interact with the UI.

[Anonymous]

Hi @Shane Piesik​ 

Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.

Please help us select the best solution by clicking on "Select As Best" if it does.

Your feedback will help us ensure that we are providing the best possible service to you. Thank you!