MLflow Model Serving latency expectations

[User16826992666]

What kind of latency should I expect when using the built in model serving capability in MLflow. Evaluating whether it would be a good fit for our use case

[sajith_appukutt · Accepted solution]

What are your throughput requirements in addition to latency. Currently this is in private preview and databricks recommends this only for low throughput and non-critical applications. However, as it move towards GA, this would change. Please get in touch with the databricks accounts team