SQL AI functions not working

[Michael_Galli]

I am working in the following environment:

- New Databricks workspsace with UC enabled
- Azure, region west-europe
- Cluster: Interactive (DBR 15.4), or SQL Warehouse (Serverless)

Running for example:

SELECT ai_gen('Generate a concise, cheerful email title for a summer bike sale with 20% discount');

Error:
[REMOTE_FUNCTION_MISSING_REQUIREMENTS_ERROR] Failed to evaluate the ai_gen SQL function because 'Databricks Foundation Models API is not available for your workspace'. Check requirements in https://docs.databricks.com/en/machine-learning/foundation-models/index.html. Please fix the problem indicated in the error message and retry the query again. SQLSTATE: 57012

Any idea what might be the problem? Thanks!

[jessysantos]

Hello @Michael_Galli 

The ai_gen uses the Mixtral-8x7B Instruct as its underlying Foundation model, which powers this and other AI functions as described in this doc: https://docs.databricks.com/en/sql/language-manual/functions/ai_gen.html#requirements

According to the error message you shared, it appears that your workspace does not have the Foundation Model APIs enabled. You should be able to verify this by checking your Serving page, where you should see the following endpoints:

You can check more information regarding these endpoints in this doc: https://docs.databricks.com/en/machine-learning/foundation-models/index.html#pay-per-token-foundatio...

If you do not see these endpoint in the Serving UI, please reach out to your Databricks account team.

Best regards,

Jéssica Santos

[Michael_Galli]

Hi Jéssica,
I only see 2 serving endpoints in Azure that come as default:

In our AWS workspace, we only have "GTE Large (En)"..
So this is nothing we can fix by ourselves, right? I will reach out to    our Databricks account then..

[Michael_Galli]

@jessysantos 
I just checked my private Azure subscription where I also have a Databricks instance set up in westeurope region.
That region is supposed to be fully supported for the foundational models.
But even there I only see the two serving endpoints above, not the full list.
Same in the workspace of our enterprise customer, also located in west europe!
So I think this is a general problem..

[Michael_Galli]

still waiting for response from @jessysantos ..

[jessysantos]

Hello @Michael_Galli 
Sorry for the delay in my response. Unfortunately, only the two models you mentioned, Meta Llama 3.1 70B Instruct and GTE Large (En), are currently available in the EU Geo. Our team is working to make more models available soon for other European regions on AWS and Azure cloud. There's no action required from your end, as these releases are handled by the Databricks side.
The good news is that some AI functions, such as the ai_query function, have an endpointName argument that allows you to specify the available models you currently have. For example, you can specify the Meta Llama 3.1 70B Instruct model. To get more details, please refer to the documentation for this specific AI function: https://docs.databricks.com/en/sql/language-manual/functions/ai_query.html#syntax

 Best regards, 

Jéssica Santos

[Michael_Galli · Accepted solution]

Thanks for the info.
As we are want to prepair a demo of all the sql ai functions, can you tell us what AWS and Azure regions currently support all required Foundation models?