topic Save VM cost when using Rest API deploying models for online inference in Machine Learning

Save VM cost when using Rest API deploying models for online inference

jhonw901227 — Thu, 09 Jun 2022 21:31:46 GMT

ADB allows us to deploy the models for online inference through a REST API. To that aim ADB creates a VM dedicated to serve a specific model. Data Scientist can create and deploy several models for testing online inference, thus the cost can rapidly grow uncontroled and unnecessarily because those newly created VMs are running all the time.

It is possible to shut them down after certain period of inactivity?

Re: Save VM cost when using Rest API deploying models for online inference

Hernandez855 — Tue, 14 Jun 2022 09:32:05 GMT

@John Wilmar Herrera Gil , See https://community.databricks.com/s/question/0D58Y00008oTDBFSA4/expose-low-latency-apis-from-deltalake-for-mobile-apps-and-microservices H&R Block Login

Re: Save VM cost when using Rest API deploying models for online inference

Anonymous — Wed, 17 Aug 2022 17:30:16 GMT

Hey there @John Wilmar Herrera Gil

Hope all is well!

Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help.

We'd love to hear from you.

Thanks!

Re: Save VM cost when using Rest API deploying models for online inference

jhonw901227 — Wed, 17 Aug 2022 18:51:36 GMT

Thanks you can close this, the solution is wait until new databricks version is delivered.

The new version has new features for model serving using AKS in the background.

Thanks

Re: Save VM cost when using Rest API deploying models for online inference

Anonymous — Thu, 18 Aug 2022 14:46:47 GMT

Hey @John Wilmar Herrera Gil

Thank you so much for getting back to us.

We really appreciate your time.

Wish you a great Databricks journey ahead!