Getting "The given endpoint does not exist" for LLM endpoint though I've access to it

Iris12 — Wed, 09 Apr 2025 12:01:20 GMT

When trying out this LLM RAG notebook steps I'm getting this error:

"The given endpoint does not exist, please retry after checking the specified model and version deployment exists"

This notebook originally referred to databricks-meta-llama-3-1-70b-instruct. After this error, I verified Models tab and found it is not available anymore. So I replaced that with meta_llama_v3_1_70b, meta_llama_v3_1_70b_instruct and

system.ai.meta_llama_v3_1_70b in the notebook but non of them work, I keep getting the same error. Also, I gave explicit apply, execute, manage permissions to my Principal user. Any pointers are greatly appreciated.

# For this first basic demo, we'll keep the configuration as a minimum. In real app, you can make all your RAG as a param (such as your prompt template to easily test different prompts!)

#"llm_model_serving_endpoint_name": "databricks-meta-llama-3-1-70b-instruct",

chain_config = {

"llm_model_serving_endpoint_name": "system.ai.meta_llama_v3_1_70b", # the foundation model we want to use

"vector_search_endpoint_name": VECTOR_SEARCH_ENDPOINT_NAME, # the endoint we want to use for vector search

"vector_search_index": f"{catalog}.{db}.databricks_documentation_vs_index",

"llm_prompt_template": """You are an assistant that answers questions. Use the following pieces of retrieved context to answer the question. Some pieces of context may be irrelevant, in which case you should not use them to form the answer.\n\nContext: {context}""",

}

Re: Getting "The given endpoint does not exist" for LLM endpoint though I've access to it

Isi — Sun, 13 Apr 2025 14:12:56 GMT

Hey @Iris12 ,

Not sure if this will fix your issue, but I’ve seen similar errors caused by permission mismatches.
Make sure that the user or service principal you’re using has the correct permissions both in the Models and Serving tab.
Also, double-check that the endpoint name you’re referencing in the notebook matches exactly

Hope that helps, 🙂
Isi

topic Getting "The given endpoint does not exist" for LLM endpoint though I've access to it in Generative AI

Getting "The given endpoint does not exist" for LLM endpoint though I've access to it

Re: Getting "The given endpoint does not exist" for LLM endpoint though I've access to it