Hello Everyone!
I've been spending a lot of time lately thinking about something that keeps coming up in almost every GenAI project I touch — how do you actually know if your model is working well? Not just in demos, but in production, day after day.
So I sat down and jotted down some of my learnings around effective model evaluation techniques for GenAI applications using the Databricks ecosystem. What does good evaluation actually look like? Why do your old ML metrics (Precision, Recall, MAE, MAPE) still matter more than you think? And how do you build a continuous eval loop that catches problems before your users do?
https://medium.com/@vinu2433/evaluating-genai-applications-the-right-way-4def3276018e?postPublishedT...
This blog walks through the full evaluation stack — from classification and regression metrics on your retrieval and extraction layers, all the way to LLM-as-a-judge and RAG-specific metrics like faithfulness and context recall — with real Databricks code and MLflow integration throughout.
In upcoming posts, we'll go deeper into prompt engineering strategies, production monitoring patterns, and building eval pipelines at scale on Databricks. Stay tuned, and I'd love to hear how your teams are approaching evals in the comments below!
