09-09-2021 10:53 PM
How to deploy real-time model on databricks at scale? Right now, The model serving is very limited to 20 requests per second. Also, There are no model monitoring framework/graphs like the one's provided with AzureML or Sagemaker frameworks.
10-10-2021 09:29 AM
I believe the next update to serving will include 1, not 2 (this is still within a Databricks workspace in a region). I don't think multi-model endpoints are on the roadmap next.
How does Airflow integration relate?
09-09-2021 11:34 PM
You might wanna look into MLFlow.
But as far as the deployment of models goes, MLFlow only does local REST APIs afaik.
Added to that you can also deploy to AzureML or Sagemaker.
Not sure what Databricks's plans are on the deployment part. I think they probably will go for out of the box integration with existing platforms, but Databricks people in here might shine a light on this.
09-09-2021 11:47 PM
https://docs.databricks.com/applications/mlflow/model-serving.html
I guess I am not up to date anymore.
09-10-2021 01:48 AM
@Werner Stinckens :
The real time capability is not yet scalable, but I have heard about an update to this in August product roadmap where databricks team have bifurcated the serving layer into 2 parts (Batch and Real-time). Not sure how much scalability is improved.
Also, There is nothing around model monitoring which is a big challenge while going to real-time model serving architecture.
09-10-2021 02:06 AM
Agree.
Which is why at my company we look at Azure ML.
09-10-2021 05:06 AM
For real time serving probably you will have to look into container services with Kubernetes. And agree deployed through Azure ML
09-10-2021 10:39 PM
It's accurate that the current Databricks model serving product has limitations regarding scalability.
That being said, MLflow has built-in deployment tools for serving products, including cloud services and open source alternatives.
We do have improvements to both our serving product regarding scalability AND monitoring on our roadmap. Happy to discuss if you are interested!
09-26-2021 11:08 PM
@Clemens Mewald : Thanks for your response.
I have heard about serving 2.0 . Would you be able to provide a rough timeline on when it will be available?
Does it include the below requirements:
Also, When will the apache airflow native integration would be available to use on databricks?
10-10-2021 09:29 AM
I believe the next update to serving will include 1, not 2 (this is still within a Databricks workspace in a region). I don't think multi-model endpoints are on the roadmap next.
How does Airflow integration relate?
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group