Azure OpenAI v1 API support for External Model Serving / Mosaic AI Gateway?

[mbecker]

Hi,

I’m setting up an external model serving endpoint for Azure OpenAI through Databricks Model Serving / Mosaic AI Gateway, and I’m trying to understand whether the newer (more than a year old at this point) Azure OpenAI v1 API is currently supported.

In the built-in OpenAI / Azure OpenAI provider configuration, Azure OpenAI mode still asks for:

• OpenAI API base
• OpenAI deployment name
• OpenAI API version

That seems aligned with the older Azure OpenAI pattern, where requests use a dated API version such as 2025-04-01-preview.

However, Microsoft’s newer Azure OpenAI v1 API uses a base URL like:

https://<resource>.openai.azure.com/openai/v1/

and no longer requires a dated api-version parameter for the v1 API.

A few questions:

1. Does the built-in Databricks Azure OpenAI external model provider support v1 as the OpenAI API version, or is that field expected to be a dated Azure API version only?
2. If v1 is supported, what should the fields look like?
• Should OpenAI API base be the resource root, like https://<resource>.openai.azure.com
• or the v1 base, like https://<resource>.openai.azure.com/openai/v1?
3. If the built-in Azure OpenAI provider does not support the new v1 URL shape, is the recommended workaround to use Custom Provider with a full endpoint URL such as:

https://<resource>.openai.azure.com/openai/v1/chat/completions

and API key authentication using the api-key header?

1. Is there a roadmap to add first-class support for Azure OpenAI v1 in External Model Serving / Mosaic AI Gateway?
2. Related: does External Model Serving currently support only Chat Completions-style endpoints, or is /openai/v1/responses support planned as well?

If anyone from Databricks or the community has a working example config for Azure OpenAI v1, either through the built-in Azure OpenAI provider or Custom Provider, I’d appreciate it.

Thanks!

[frankieseabrook]

Short answer: based on the current Databricks docs, I would treat the built-in Azure OpenAI external model provider as expecting the older Azure OpenAI configuration shape, not the newer `/openai/v1/` shape.

The key clue is that the Databricks Azure OpenAI provider still documents `openai_api_version` as required and says it is “specified by a date”. The example also uses:

"openai_api_base": "https://my-azure-openai-endpoint.openai.azure.com",
"openai_deployment_name": "my-gpt-35-turbo-deployment",
"openai_api_version": "2023-05-15"

So I would not assume that setting openai_api_version = "v1" is supported unless Databricks confirms it.

What I would try:

1. For the built-in Azure OpenAI provider, use the resource root as the base URL:

"openai_api_type": "azure",
"openai_api_base": "https://<resource>.openai.azure.com",
"openai_deployment_name": "<deployment-name>",
"openai_api_version": "<dated-api-version>"

1. If you specifically need the Azure OpenAI v1 URL shape, use the custom provider instead, because Databricks documents custom for OpenAI-compatible providers not directly supported by Databricks.

For chat completions, that would likely look like:

"external_model": {
  "name": "<deployment-name>",
  "provider": "custom",
  "task": "llm/v1/chat",
  "custom_provider_config": {
    "custom_provider_url": "https://<resource>.openai.azure.com/openai/v1/chat/completions",
    "api_key_auth": {
      "key": "api-key",
      "value": "{{secrets/<scope>/<azure-openai-key>}}"
    }
  }
}

Why this works:

Microsoft’s v1 Azure OpenAI API uses:

https://<resource>.openai.azure.com/openai/v1/

and no longer requires a dated api-version parameter. Databricks’ built-in Azure OpenAI config, however, still documents a dated openai_api_version, deployment name, and Azure resource base URL. So the custom provider is the cleaner fit if you want to call the v1-style endpoint directly.

Caveat:

I would be careful with /openai/v1/responses. Databricks external model tasks are documented around llm/v1/completions, llm/v1/chat, and llm/v1/embeddings, and the custom provider must be OpenAI API compatible for those task shapes. I do not see /responses documented as a supported external model task shape today, so I would expect /openai/v1/chat/completions to be the safer custom-provider target.

For roadmap questions, such as first-class Azure OpenAI v1 support or Responses API support, I think this needs a Databricks product/support confirmation.

References:

• Databricks external models docs: https://docs.databricks.com/aws/en/generative-ai/external-models/
• Microsoft Azure OpenAI v1 API docs: https://learn.microsoft.com/en-us/azure/foundry/openai/api-version-lifecycle?tabs=python