The inference table is not updated
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-07-2025 03:21 PM
Hi,
I am deploying a model with following code:
w = WorkspaceClient()
model_cfg = {
"entity_name": uc_model,
"entity_version": str(version),
"workload_type": "CPU",
"workload_size": "Small",
"scale_to_zero_enabled": True
}
ai_gateway_config = AiGatewayConfig(
inference_table_config=AiGatewayInferenceTableConfig(
catalog_name=catalog,
schema_name=schema,
enabled=True
)
)
served_model = ServedEntityInput.from_dict(model_cfg)
endpoint_config = EndpointCoreConfigInput(served_entities=[served_model])
try:
w.serving_endpoints.get(endpoint_name)
w.serving_endpoints.update_config(
name=endpoint_name,
served_entities=[served_model]
)
except NotFound:
w.serving_endpoints.create(
name=endpoint_name,
ai_gateway=ai_gateway_config,
config=endpoint_config,
route_optimized=True
)
It creates the inference table, the endpoint makes predictions, but it doesn't log the request and response. Could someone point me what I am missing? Thank you.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-23-2025 02:32 AM
Hi Dharma,
As mentioned in the documentation, Inference table log delivery is currently best effort, but logs are usually available within 1 hour of a request.
Please try to query the inference tables after waiting for an hour.
There are certain scenarios where the records don't show up in the inference tables. For example, consider a scenario where the request simply is not authorized with Databricks credentials—there is no way for us to log this, as we cannot admit the request into the Databricks system. This extends to rate limits and a few other cases as well. TLDR: We generally guarantee all scored requests are logged, but various issues can occur before model scoring/inference.
I hope this help 🙂