topic Re: SQL AI functions not working in Generative AI

SQL AI functions not working

Michael_Galli — Mon, 19 Aug 2024 18:23:48 GMT

I am working in the following environment:

- New Databricks workspsace with UC enabled
- Azure, region west-europe
- Cluster: Interactive (DBR 15.4), or SQL Warehouse (Serverless)

Running for example:

SELECT ai_gen('Generate a concise, cheerful email title for a summer bike sale with 20% discount');

Error:
[REMOTE_FUNCTION_MISSING_REQUIREMENTS_ERROR] Failed to evaluate the ai_gen SQL function because 'Databricks Foundation Models API is not available for your workspace'. Check requirements in https://docs.databricks.com/en/machine-learning/foundation-models/index.html. Please fix the problem indicated in the error message and retry the query again. SQLSTATE: 57012

Any idea what might be the problem? Thanks!

Re: SQL AI functions not working

jessysantos — Mon, 19 Aug 2024 20:32:05 GMT

Hello @Michael_Galli

The ai_gen uses the Mixtral-8x7B Instruct as its underlying Foundation model, which powers this and other AI functions as described in this doc: https://docs.databricks.com/en/sql/language-manual/functions/ai_gen.html#requirements

According to the error message you shared, it appears that your workspace does not have the Foundation Model APIs enabled. You should be able to verify this by checking your Serving page, where you should see the following endpoints:

You can check more information regarding these endpoints in this doc: https://docs.databricks.com/en/machine-learning/foundation-models/index.html#pay-per-token-foundation-model-apis

If you do not see these endpoint in the Serving UI, please reach out to your Databricks account team.

Best regards,

Jéssica Santos

Re: SQL AI functions not working

Michael_Galli — Tue, 20 Aug 2024 00:54:49 GMT

Hi Jéssica,
I only see 2 serving endpoints in Azure that come as default:

In our AWS workspace, we only have "GTE Large (En)"..
So this is nothing we can fix by ourselves, right? I will reach out to our Databricks account then..

Re: SQL AI functions not working

Michael_Galli — Tue, 20 Aug 2024 07:46:44 GMT

@jessysantos
I just checked my private Azure subscription where I also have a Databricks instance set up in westeurope region.
That region is supposed to be fully supported for the foundational models.
But even there I only see the two serving endpoints above, not the full list.
Same in the workspace of our enterprise customer, also located in west europe!
So I think this is a general problem..

Re: SQL AI functions not working

Retired_mod — Tue, 20 Aug 2024 12:36:15 GMT

Hi @Michael_Galli, Thanks for reaching out! Please review the response and let us know if it answers your question. Your feedback is valuable to us and the community.

If the response resolves your issue, kindly mark it as the accepted solution. This will help close the thread and assist others with similar queries.

We appreciate your participation and are here if you need further assistance!

Re: SQL AI functions not working

Michael_Galli — Tue, 20 Aug 2024 12:46:00 GMT

still waiting for response from @jessysantos ..

Re: SQL AI functions not working

jessysantos — Tue, 20 Aug 2024 18:35:42 GMT

Hello @Michael_Galli
Sorry for the delay in my response. Unfortunately, only the two models you mentioned, Meta Llama 3.1 70B Instruct and GTE Large (En), are currently available in the EU Geo. Our team is working to make more models available soon for other European regions on AWS and Azure cloud. There's no action required from your end, as these releases are handled by the Databricks side.
The good news is that some AI functions, such as the ai_query function, have an endpointName argument that allows you to specify the available models you currently have. For example, you can specify the Meta Llama 3.1 70B Instruct model. To get more details, please refer to the documentation for this specific AI function: https://docs.databricks.com/en/sql/language-manual/functions/ai_query.html#syntax

Best regards,

Jéssica Santos

Re: SQL AI functions not working

Michael_Galli — Wed, 21 Aug 2024 06:38:26 GMT

Thanks for the info.
As we are want to prepair a demo of all the sql ai functions, can you tell us what AWS and Azure regions currently support all required Foundation models?