Hello everyone,
I have an Azure Databricks subscription with my company, and I want to use external LLMs in databricks, like claude-3 or gemini. I managed to create a serving endpoint for Anthropic and I am able to use claude 3.
But I want to use a Gemini model through Databricks. I created a Vertex AI account and took a free subscription. I created an API key for my account.
And then I followed the exact steps needed to create an external service endpoint :
- Configure Databricks with the URL of my ressource and my personal token.
- Creation of a scope named "google-scope" and of a key named "dematbl", dematbl being the name of my project.
I then insert the key value that I got from Vertex AI.
- I go on the creation of a serving endpoint page and I write all the informations (Name of supplier, task, reference of API secret key, name of external model).
I did the exact same steps with Claude-3-sonnet and It works perfectly fine, but It does not work with Gemini. I receive the following error when using the playground :
{"error_code":"INVALID_PARAMETER_VALUE","message":"INVALID_PARAMETER_VALUE: Failed to generate access token for Google VertexAI. Please check the private key for the external model gemini-pro."}Thank you for your help.
LÊo.
