โ09-09-2021 10:53 PM
How to deploy real-time model on databricks at scale? Right now, The model serving is very limited to 20 requests per second. Also, There are no model monitoring framework/graphs like the one's provided with AzureML or Sagemaker frameworks.
โ10-10-2021 09:29 AM
I believe the next update to serving will include 1, not 2 (this is still within a Databricks workspace in a region). I don't think multi-model endpoints are on the roadmap next.
How does Airflow integration relate?
โ09-09-2021 11:34 PM
You might wanna look into MLFlow.
But as far as the deployment of models goes, MLFlow only does local REST APIs afaik.
Added to that you can also deploy to AzureML or Sagemaker.
Not sure what Databricks's plans are on the deployment part. I think they probably will go for out of the box integration with existing platforms, but Databricks people in here might shine a light on this.
โ09-09-2021 11:47 PM
https://docs.databricks.com/applications/mlflow/model-serving.html
I guess I am not up to date anymore.
โ09-10-2021 01:48 AM
@Werner Stinckensโ :
The real time capability is not yet scalable, but I have heard about an update to this in August product roadmap where databricks team have bifurcated the serving layer into 2 parts (Batch and Real-time). Not sure how much scalability is improved.
Also, There is nothing around model monitoring which is a big challenge while going to real-time model serving architecture.
โ09-10-2021 02:06 AM
Agree.
Which is why at my company we look at Azure ML.
โ09-10-2021 05:06 AM
For real time serving probably you will have to look into container services with Kubernetes. And agree deployed through Azure ML
โ09-10-2021 10:39 PM
It's accurate that the current Databricks model serving product has limitations regarding scalability.
That being said, MLflow has built-in deployment tools for serving products, including cloud services and open source alternatives.
We do have improvements to both our serving product regarding scalability AND monitoring on our roadmap. Happy to discuss if you are interested!
โ09-26-2021 11:08 PM
@Clemens Mewaldโ : Thanks for your response.
I have heard about serving 2.0 . Would you be able to provide a rough timeline on when it will be available?
Does it include the below requirements:
Also, When will the apache airflow native integration would be available to use on databricks?
โ10-10-2021 09:29 AM
I believe the next update to serving will include 1, not 2 (this is still within a Databricks workspace in a region). I don't think multi-model endpoints are on the roadmap next.
How does Airflow integration relate?
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโt want to miss the chance to attend and share knowledge.
If there isnโt a group near you, start one and help create a community that brings people together.
Request a New Group