Hello @Khasim_1 , I took a look at both internal and external documentation and here is what I found.
Building on @robert28hull's point, I'd treat the gateway as the default path for dev, test, and production rather than a production-only gate. What changes by environment is the controls, not the path. One naming note: the docs now call this Unity Gateway, since it's built on Unity Catalog and governs models, external providers, and MCP tools with the same privileges you use for tables. Overview: https://docs.databricks.com/aws/en/ai-gateway/ai-governance
The reason to go "everywhere" is credentials. Once dev teams can bypass the gateway, provider keys end up in notebooks and repos and you don't get them back. Register each external provider once as a model provider service in Unity Catalog (a securable with a three-part name like main.default.openai_dev), and callers authenticate with their Databricks token, never the provider secret: https://docs.databricks.com/aws/en/ai-gateway/model-provider-services
Agility comes from configuration:
- Separate provider services or model services for dev and prod, with looser limits in dev and broader quotas only after an application is promoted. Access is a GRANT statement, so onboarding is a one-liner rather than a ticket.
- Per-user and per-group rate limits on shared endpoints, so one runaway loop doesn't starve everyone: https://docs.databricks.com/aws/en/ai-gateway/rate-limits
- Budgets with alert thresholds and hard caps, so you can say yes to experimentation without a surprise invoice: https://docs.databricks.com/aws/en/ai-gateway/budgets
- Foundation Model APIs in system.ai need no setup at all, which is the fastest path for a lot of dev work and it's already governed.
On model-specific features, the trade-off is smaller than it used to be. Managed paths expose each provider's own API (OpenAI Chat Completions and Responses, Anthropic Messages, Gemini generateContent), so you point the provider's SDK at the gateway base URL, add a header naming the provider service, and use the native request format with guardrails, rate limits, and usage tracking applied on the way through. For anything a managed path doesn't cover (OpenAI files or batch, for example), you can enable "Forward all URL paths" and the request goes to the provider unchanged. The catch: token and cost tracking, token-based rate limits, model access control, and service policies don't apply to passthrough requests. So keep the application contract generic for the common path, allow passthrough on dev provider services only, and keep it off in prod. That gives you the split robert28hull described without anyone going around the gateway: https://docs.databricks.com/aws/en/ai-gateway/query-model-provider-services
A practical promotion gate: no production workload until its model service has an owner, an approved data and PII policy, rate and spend limits, usage logging turned on, and an evaluation result for the selected model. Also confirm feature support for the exact endpoint type, since capabilities differ across external, foundation, custom-model, and agent endpoints: https://docs.databricks.com/aws/en/ai-gateway/create-model-services
Last thought on the Medallion Architecture framing. The gateway sits at the consumption layer rather than in the Bronze/Silver/Gold flow, but payload logging writes prompts and responses to Delta tables in Unity Catalog (inference tables), and those become raw input you can refine for evaluation, drift monitoring, or fine-tuning datasets. Design for that early, because it's harder to bolt on later: https://docs.databricks.com/aws/en/ai-gateway/inference-tables
Regards, Louis.
