Feature Store Benchmarks

FedeRaimondi — Fri, 11 Jul 2025 12:27:01 GMT

We are currently planning to create feature tables to serve machine learning models in our organization.

I am struggling to find interesting benchmarks on Databricks Feature Store performances vs using directly Delta Tables. It would also be interesting the different results for batch inference and usage of online feature store for real time inference.

Throwing the topic in the community to see if someone in Databricks or other clients have tried running some tests.

Re: Feature Store Benchmarks

WiliamRosa — Sun, 17 Aug 2025 13:55:10 GMT

We’re also exploring this internally and found very limited public benchmarks comparing Databricks Feature Store to directly using Delta Tables. That said, the open-source project featurestore-benchmarks provides a framework to evaluate offline and online feature store performance across platforms, which could be adapted for Databricks:
https://github.com/featurestoreorg/featurestore-benchmarks

Additionally, Hopsworks published some academic benchmarks comparing their feature store to Databricks, SageMaker, and Vertex AI. While results may not generalize fully, they provide useful performance reference points:
https://www.hopsworks.ai/news/redefining-feature-stores-with-class-leading-performance

From community discussions, Delta Tables may be sufficient for batch inference, but Feature Store provides added value for point-in-time joins, versioning, and online inference:
https://www.reddit.com/r/mlops/comments/14fj1o7
https://www.reddit.com/r/mlops/comments/17p0w7h

We’re considering setting up our own benchmarks using these tools. Would be great to hear if others have done similar testing on Databricks.

topic Re: Feature Store Benchmarks in Machine Learning

Feature Store Benchmarks

Re: Feature Store Benchmarks