When trying out this LLM RAG notebook steps I'm getting this error:
"The given endpoint does not exist, please retry after checking the specified model and version deployment exists"
This notebook originally referred to databricks-meta-llama-3-1-70b-instruct. After this error, I verified Models tab and found it is not available anymore. So I replaced that with meta_llama_v3_1_70b, meta_llama_v3_1_70b_instruct and
system.ai.meta_llama_v3_1_70b in the notebook but non of them work, I keep getting the same error. Also, I gave explicit apply, execute, manage permissions to my Principal user. Any pointers are greatly appreciated.
# For this first basic demo, we'll keep the configuration as a minimum. In real app, you can make all your RAG as a param (such as your prompt template to easily test different prompts!)
#"llm_model_serving_endpoint_name": "databricks-meta-llama-3-1-70b-instruct",
chain_config = {
"llm_model_serving_endpoint_name": "system.ai.meta_llama_v3_1_70b", # the foundation model we want to use
"vector_search_endpoint_name": VECTOR_SEARCH_ENDPOINT_NAME, # the endoint we want to use for vector search
"vector_search_index": f"{catalog}.{db}.databricks_documentation_vs_index",
"llm_prompt_template": """You are an assistant that answers questions. Use the following pieces of retrieved context to answer the question. Some pieces of context may be irrelevant, in which case you should not use them to form the answer.\n\nContext: {context}""",
}
