Is it possible to configure the evaluation runs overview?

lkt1 — Tue, 14 Jul 2026 08:06:15 GMT

I would like to be able to add columns to this overview, if possible. For example, the latency of the model is available, when I click on a run, but I would like to see the average latency, similarly to, how the score is shown. Of course, I could add a new score, which is latency, but I'd like to know if I can use the already existing data.

Along with this, I would like to be able to configure it such, that I can see the percentiles and medians of my score. Is any of this possible to do in the overview?

Re: Is it possible to configure the evaluation runs overview?

Louis_Frolio — Wed, 15 Jul 2026 19:48:46 GMT

Greetings @lkt1 , Good set of questions here, and you're already thinking about this the right way: reuse the data you've got rather than bolt on redundant scorers. You're really asking two things, and they have different answers, so let me take them one at a time.

Short version up front: the Evaluation runs overview table itself isn't documented as configurable for arbitrary columns, but everything you're after is reachable, and you don't need a new latency scorer to get the average.

Percentiles and medians of your score

This is the direct one. Scorers take an aggregations argument, and the default is mean only, which is why you're seeing a single number. You can ask for more when you define the scorer:

@scorer(aggregations=["mean", "median", "p90"])

The built-in options are min, max, mean, median, variance, and p90. If you need something else (p50, p95, p99), pass a callable that takes a list of values and returns one number. Each aggregation gets logged as its own run metric, roughly scorer_name/median and scorer_name/p90.

One catch to save you some trouble: these are computed when mlflow.genai.evaluate() runs, so the UI won't go back and add a median to a run that only logged a mean. If you'd rather not re-run, pull the per-row feedback with mlflow.search_traces() and compute the stats in pandas.

Average latency (you already have it)

You don't need a new scorer for this one. An evaluation run auto-logs the agent metrics (latency and token counts) as run-level aggregates, so the average latency is already sitting there as data. If you specifically want median or p90 latency, add a small custom scorer that returns the trace execution time and set aggregations=["mean", "median", "p90"] on it, same pattern as above.

Getting them into the overview

Once those values are logged as run metrics, take a gander at the Columns control on the Evaluation runs table (the same button in your screenshot). I'll be straight with you: the docs don't spell out exactly which metrics that picker exposes for this particular view, so confirm it actually surfaces the latency and percentile columns rather than taking my word for it.

If you want this for reporting or a dashboard

The MLflow system tables are the better tool for anything persistent. system.mlflow.run_metrics_history holds every metric logged on a run, and system.mlflow.runs_latest holds the aggregated min, max, and latest per metric, so you can query across runs in SQL and build an AI/BI dashboard on top. One thing to keep in mind: these tables reflect the metrics you logged, so they'll give you the median or p90 you configured above, not a percentile you never computed. Three practical caveats:

They're in Public Preview.
Access is admin-gated by default (an account admin grants USE and SELECT on system.mlflow).
Data only goes back to September 2, 2025.

The takeaway

Configure the aggregations you want at evaluation time, use the auto-logged latency for the average, and reach for the system tables when you need flexible reporting. One prerequisite before any of this: make sure you're on mlflow[databricks]>=3.1, since the aggregations API and the current GenAI eval UI came in with the MLflow 3 integration.

Hope that helps

Regards, Louis

Re: Is it possible to configure the evaluation runs overview?

henry_collins — Wed, 15 Jul 2026 21:37:41 GMT

I don’t think the evaluation runs overview currently supports adding custom columns or directly displaying existing metrics like latency percentiles/medians. The usual workaround is to create custom scores or export the evaluation data and analyze it separately. It would be a useful feature though, especially for tracking operational metrics alongside quality scores.

topic Re: Is it possible to configure the evaluation runs overview? in Warehousing & Analytics

Is it possible to configure the evaluation runs overview?

Re: Is it possible to configure the evaluation runs overview?

Percentiles and medians of your score

Average latency (you already have it)

Getting them into the overview

If you want this for reporting or a dashboard

The takeaway

Re: Is it possible to configure the evaluation runs overview?