Inference Lakehouse Monitor , how to create a monitor on more than one prediction

noah_sunny — Thu, 19 Sep 2024 10:19:17 GMT

Currently, prediction_col only accept string type, but I want to monitor a list of predictions, like in quantile regression. Any tips?

info = w.quality_monitors.create(

table_name=TABLE_NAME,

inference_log=MonitorInferenceLog(

granularities=GRANULARITIES,

timestamp_col=TIMESTAMP_COL,

model_id_col=MODEL_ID_COL, # Model version number

prediction_col=PREDICTION_COL,

problem_type=PROBLEM_TYPE,

label_col=LABEL_COL # Optional

baseline_table_name=BASELINE_TABLE,

slicing_exprs=SLICING_EXPRS,

output_schema_name=f"{CATALOG}.{SCHEMA}",

assets_dir=ASSETS_DIR

)

Re: Inference Lakehouse Monitor , how to create a monitor on more than one prediction

AustinZaccor — Tue, 05 May 2026 01:27:09 GMT

Unfortunately this can only take a single column. To monitor a list of predictions for quantile regression, you need to flatten the list into distinct columns before creating the monitor.

Re: Inference Lakehouse Monitor , how to create a monitor on more than one prediction

Sureshkrishna — Fri, 26 Jun 2026 06:03:29 GMT

This is a known limitation of the Databricks Lakehouse Monitoring API. Here's what you need to know and the workarounds:

The Limitation:
"prediction_col" must be a double type (not a string, and not a list/array)
Reference : https://medium.com/marvelous-mlops/streamlining-ml-model-monitoring-with-databricks-lakehouse-and-inference-tables-f0a99ab03291

The API only accepts a single scalar column. There's no native support for multi-output predictions like quantile regression.
Workaround Options:
1. Create separate monitors per quantile (Recommended) : Unpivot your quantile predictions into separate tables, each with its own scalar "prediction_col", and create one monitor per quantile

e.g., for quantiles [0.1, 0.5, 0.9]
for quantile, col_name in [("q10", "pred_q10"), ("q50", "pred_q50"), ("q90", "pred_q90")]:
spark.sql(f"""
CREATE OR REPLACE TABLE {CATALOG}.{SCHEMA}.inference_log_{quantile}
AS SELECT *, {col_name} AS prediction FROM {TABLE_NAME}
""")
w.quality_monitors.create(
table_name=f"{CATALOG}.{SCHEMA}.inference_log_{quantile}",
inference_log=MonitorInferenceLog(
granularities=GRANULARITIES,
timestamp_col=TIMESTAMP_COL,
model_id_col=MODEL_ID_COL,
prediction_col="prediction", 
problem_type=MonitorInferenceLogProblemType.PROBLEM_TYPE_REGRESSION,
label_col=LABEL_COL,
),
output_schema_name=f"{CATALOG}.{SCHEMA}",
assets_dir=f"{ASSETS_DIR}/{quantile}",
)

2. Use "custom_metrics" for the extra quantiles

Keep one quantile as the primary "prediction_col" (e.g., median), and track the others via custom metrics:

from databricks.sdk.service.catalog import MonitorMetric, MonitorMetricType
custom_metrics = [
MonitorMetric(
name="mean_pred_q10",
input_columns=["pred_q10"],
definition="avg(:pred_q10)",
output_data_type="double",
type=MonitorMetricType.CUSTOM_METRIC_TYPE_AGGREGATE,
),
MonitorMetric(
name="mean_pred_q90",
input_columns=["pred_q90"],
definition="avg(:pred_q90)",
output_data_type="double",
type=MonitorMetricType.CUSTOM_METRIC_TYPE_AGGREGATE,
),
]

w.quality_monitors.create(
table_name=TABLE_NAME,
inference_log=MonitorInferenceLog(
prediction_col="pred_q50", ]
...
),
custom_metrics=custom_metrics,
...
)

3. Use Snapshot analysis instead

If you don't need the inference-specific accuracy metrics (MSE, etc.), switch to a Snapshot monitor. It will profile all your quantile columns as regular numeric columns and still compute drift/distribution stats across all of them no "prediction_col" constraint.

topic Re: Inference Lakehouse Monitor , how to create a monitor on more than one prediction in Khoros Community Forums Support (Not for Databricks Product Questions)

Inference Lakehouse Monitor , how to create a monitor on more than one prediction

Re: Inference Lakehouse Monitor , how to create a monitor on more than one prediction

Re: Inference Lakehouse Monitor , how to create a monitor on more than one prediction