"error_code":"INVALID_PARAMETER_VALUE","message":"INVALID_PARAMETER_VALUE: Failed to generate access

[Leo69]

Hello everyone,

I have an Azure Databricks subscription with my company, and I want to use external LLMs in databricks, like claude-3 or gemini. I managed to create a serving endpoint for Anthropic and I am able to use claude 3.

But I want to use a Gemini model through Databricks. I created a Vertex AI account and took a free subscription. I created an API key for my account.

And then I followed the exact steps needed to create an external service endpoint :

- Configure Databricks with the URL of my ressource and my personal token.

- Creation of a scope named "google-scope" and of a key named "dematbl", dematbl being the name of my project.
I then insert the key value that I got from Vertex AI.

- I go on the creation of a serving endpoint page and I write all the informations (Name of supplier, task, reference of API secret key, name of external model).

I did the exact same steps with Claude-3-sonnet and It works perfectly fine, but It does not work with Gemini. I receive the following error when using the playground : 

{"error_code":"INVALID_PARAMETER_VALUE","message":"INVALID_PARAMETER_VALUE: Failed to generate access token for Google VertexAI. Please check the private key for the external model gemini-pro."}

Thank you for your help.

Léo.

[Kaniz_Fatma]

Hi @Leo69, It seems you’re encountering an issue while trying to use the Gemini model through Databricks.

Let’s troubleshoot this together!

First, let’s review some important information about external models in Databricks Model Serving. External models are third-party models hosted outside of Databricks. They allow you to streamline the usage and management of various large language model (LLM) providers within your organization. Supported providers include OpenAI, Anthropic, and others¹.

Here are some steps you can take to address the error you’re encountering:

1. Verify API Key and Configuration:

   • Double-check that the API key you created for your Vertex AI account is correctly configured in Databricks.
   • Ensure that the key value you inserted matches the one provided by Vertex AI.

2. Endpoint Configuration:

   • When creating the serving endpoint for Gemini, make sure you include the necessary parameters in the endpoint configuration.
   • Specifically, ensure that you specify the correct external model name (“gemini-pro”) and any other relevant details.

3. Access Token Generation:

   • The error message indicates that there’s an issue generating an access token for Google VertexAI. This could be related to the private key for the external model.
   • Verify that the private key associated with the “gemini-pro” model is correctly configured in Databricks.

4. Provider-Specific Documentation:

   • Consult the official documentation for Google Cloud Vertex AI to understand any specific requirements or configurations needed for integrating with Databricks.
   • Check if there are any additional steps or settings related to Gemini models.

5. Logs and Diagnostics:

   • Review the logs or diagnostic information in Databricks to identify any specific errors or warnings related to the Gemini model.
   • Look for any clues that might help pinpoint the issue.

Remember that external models can be powerful tools, but proper configuration and troubleshooting are essential to ensure smooth integration. Good luck, and I hope you’re able to resolve the issue! 😊¹²³.