topic Re: getting 500 on embedding model invocation call in Generative AI

getting 500 on embedding model invocation call

tefrati — Fri, 10 Oct 2025 19:43:02 GMT

I'm getting the following error message "{"error_code": "INTERNAL_ERROR", "message": "The server received an invalid response from an upstream server."}" when making a call to bge-large-en embedding model.

Re: getting 500 on embedding model invocation call

nayan_wylde — Fri, 10 Oct 2025 20:18:09 GMT

Seems to me like a rate limit issue. Can you please confirm if the rate limit is not zero.

Re: getting 500 on embedding model invocation call

tefrati — Fri, 10 Oct 2025 22:01:56 GMT

I don't see that option on my serving endpoint.
Also, if there's a rate limit, I'd expect to receive a 429 with relevant message and not 500.

Re: getting 500 on embedding model invocation call

nayan_wylde — Fri, 10 Oct 2025 22:25:05 GMT

Can you please click on the edit AI gateway it will take you show you the rate limit and share the screenshot

Re: getting 500 on embedding model invocation call

tefrati — Fri, 10 Oct 2025 22:30:07 GMT

no rate limitation is enabled

Re: getting 500 on embedding model invocation call

nayan_wylde — Fri, 10 Oct 2025 23:03:08 GMT

Yeah your rate limit seems to be good. Can you also check the following points.

1. Use the Databricks-specific name (e.g., databricks-bge-large-en), not the Hugging Face model name. Check in Serving → Endpoints.
2. Validate Payload Format
{ "input": "text to embed" }
3. Test via Databricks UIUse Query endpoint in the Serving page. If that works, issue is client config.

Re: getting 500 on embedding model invocation call

tefrati — Sat, 11 Oct 2025 04:02:40 GMT

I have been using databricks-bge-large-en.

Not sure what is meant by client config. I've been using this model for 2 years now. Most of the time it works but today and two days ago is stopped working intermittedly.

Re: getting 500 on embedding model invocation call

tefrati — Sat, 18 Oct 2025 14:41:04 GMT

Problem was not resolved.. Any thought as to what else could have happened?

Re: getting 500 on embedding model invocation call

dkushari — Sat, 18 Oct 2025 17:03:12 GMT

Hi @tefrati - Can you please go to the model serving endpoint and click on the use drop down (as shown in the picture below) and try the simple python or sql and see if that works for you.

from openai import OpenAI import os # How to get your Databricks token: https://docs.databricks.com/en/dev-tools/auth/pat.html DATABRICKS_TOKEN = os.environ.get('DATABRICKS_TOKEN') # Alternatively in a Databricks notebook you can use this: # DATABRICKS_TOKEN = dbutils.notebook.entry_point.getDbutils().notebook().getContext().apiToken().get() client = OpenAI( api_key=DATABRICKS_TOKEN, base_url="https://e2-demo-field-eng.cloud.databricks.com/serving-endpoints" ) embeddings = client.embeddings.create( input='Your string for the embedding model goes here', model="databricks-bge-large-en" ) print(embeddings.data[0].embedding)

SELECT ai_query('databricks-bge-large-en', request => "<Please provide your input string here!>")

Re: getting 500 on embedding model invocation call

tefrati — Fri, 24 Oct 2025 03:40:30 GMT

Hi there,

I've ran the python script and got the vector back. Also, I've been using the model and the endpoint for 2 years now so no reason it shouldn't work.
However, I'm getting the same error again now. This time, Databricks select query:

SELECT

FROM

<my company>.default.`databricks-bge-large-en_payload`

where

status_code != 200

ORDER BY request_date DESC

returns numerous 502 status code (after 5 seconds execution time each time). Response is: {"error_code":"INTERNAL_ERROR","message":"The server received an invalid response from an upstream server."}

Can I finally get an answer as to why we're keep getting these unpredictable errors?

Thank you