Your read is right. The confusion comes from two products sharing a name. The ai_gateway block on a serving endpoint is the older per-endpoint feature set: usage tracking, inference tables, rate limits, and guardrails for foundation and external models. Unity AI Gateway is the newer system where Model Services, MCP Services and service policies live as Unity Catalog securables. Your endpoint has the first. Guardrails come from the second, and the second cannot wrap an agent. Taking your questions in order:
1. Can a Model Service sit in front of the agent endpoint? No.
CreateModelService accepts three destination types: pay-per-token foundation model, provisioned-throughput foundation model, external foundation model. None of them is a serving endpoint or a UC-registered model, which is why your agent never shows up in the picker. A Model Service routes to a model. It does not proxy an endpoint.
The service policies page states it outright:
Service policies apply to MCP Services, Model Services, and Model Provider Services. Agent Services are not supported.
You will find Agent Services in the catalog and assume they solve this. They register external agents for discovery and grants only. Runtime invocation is not available, and policies and rate limits are not supported on them.
2. Can the Model Service be the agent's LLM? Yes. This is the design.
Create the Model Service over your foundation or external model, attach the policies there, and point the LLM inside the graph at it:
from databricks_langchain import ChatDatabricks
llm = ChatDatabricks(
model="<catalog>.<schema>.<model_service>",
use_ai_gateway=True,
)use_ai_gateway=True switches the client's base URL from /serving-endpoints to /ai-gateway/mlflow/v1, the same endpoint the query docs show with the OpenAI SDK, and the three-level name goes through as the model field. Callers need USE CATALOG, USE SCHEMA, and EXECUTE on the service (privilege table).
One trap: anything you route through ai_query gets usage tracking and nothing else. Guardrails, rate limits, inference tables and fallbacks do not apply on that path.
3. Whole execution path, or only the model call? Only the model call, and narrower than you would guess.
A policy runs ON CALL against the request and ON RESULT against the response, and returns ALLOW, DENY, or ASK. ASK parks the interaction for human approval. The beta limits are the part to internalise:
By default each evaluation is scoped to one message, so a built-in service policy can't detect patterns that span multiple messages.
The evaluator sees only that single extracted item. It doesn't see the system prompt of the protected service or image and audio content.
You can widen the input window to recent turns for model and model provider services. Output evaluation and MCP services stay single-message. Policies decide, they do not rewrite, with one exception: system.ai.detect_sensitive_data redacts matches on model services.
Nothing in that path sees LangGraph orchestration, local Python tools, direct REST calls, or writes. If the agent calls three models and one goes through the governed service, you have covered one call.
Built-ins: system.ai.block_unsafe_content, system.ai.block_jailbreak, system.ai.block_hallucination, system.ai.detect_sensitive_data. Anything else is a SQL UDF you write that receives the interaction event and returns a decision.
4. Recommended shape
Tools are their own boundary, and MCP Services enforce it: the gateway checks EXECUTE on the service, exposes only the tools you selected (prefix get_* or exact match), and applies policies per tool call. Put ASK on anything destructive or externally visible.
Keep your own validation around the graph. With single-message scope and no view of the system prompt, a Model Service policy is a strong filter on model traffic and a weak defence for the workflow as a whole.
5. Is "AI Gateway enabled" just usage tracking on an agent endpoint? Yes.
The support matrix marks AI Guardrails Not supported for Databricks agents and for custom model endpoints. For agents, payload logging to inference tables is the only Supported feature. No setting on that endpoint gives you jailbreak or unsafe-content filtering, and service policies do not reach it either. Your usage_tracking_config block is observability.
To confirm what crosses the gateway once you rewire the LLM call:
SELECT service_name, requester, status_code, COUNT(*) AS calls FROM system.ai_gateway.usage WHERE service_type = 'MODEL_SERVICE' GROUP BY ALL ORDER BY calls DESC;
Service policies are Beta; an account admin enables them from the account console Previews page.
In short: leave the agent endpoint as the deployment surface, move the LLM call behind a Model Service, move tools behind MCP Services, and keep your own checks around the graph.
Announcements
